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I.  INTRODUCTION 


A.  BACKGROUND 

This  research  will  examine  and  evaluate  the  emergence  of  the  International 
Organization  for  Standardization,  commonly  referred  to  as  ISO,  quality  standard  for  the 
year  2000  as  it  applies  (past  and  present)  to  the  Department  of  Defense  (DoD)  quality 
management  system  (QMS)  in  procurement.  In  particular,  the  researcher  will  examine  the 
new  standard  and  its  utility  for  DoD  procurement,  focusing  on  changes  from  the  previous 
ISO  9000  series.  Leading  defense  industry  quality  managers  and  Defense  Contract 
Management  Agency  Quality  Assurance  Managers  will  provide  insight  to  the  thesis.  In 
order  to  understand  the  present  and  future  of  DoD  Quality  Assurance  and  Management, 
the  thesis  will  briefly  look  at  past  quality  assurance  policy  from  DoD  and  the  ISO. 
Likewise,  this  research  will  explore  issues  and  concerns  that  contracting  officers  and 
program  managers  in  Government  will  now  need  to  become  familiar  with  as  they  execute 
ISO  9000:2000.  The  effort  will  emphasize  how  ISO  9000:2000  fits  within  the  realm  of 
DoD  procurement  and  what  organizations  need  to  do  in  order  achieve  excellent  products 
in  a  total  quality  management  environment. 

The  past  decade  has  brought  marked  change  in  the  way  DoD  conducts 
acquisition.  A  culture  versed  in  military  specifications,  regulations,  and  standards  now 
turns  its  focus  toward  better  understanding  the  global  marketplace  and  adapting  the 
“new”  economy  culture  to  DoD  procurement.  One  such  change  is  in  DoD  management 
of  quality.  MIL-Q-STD  9858,  once  the  standard  by  which  all  DoD  contractors’  quality 
assurance  programs  were  measured,  is  now  the  least  preferred  by  DoD. 

The  February  14,  1994,  memorandum  “Use  of  Commercial  Standards  in  DoD” 
from  then  Under  Secretary  of  Defense  (USD)  John  M.  Deutch  opened  and  encouraged 
DoD  acquisition  and  contracting  professionals  to  “use  commercial  standards”  to  their 
maximum  extent  in  all  applicable  acquisition  matters.  One  memorandum  began  the 
change  of  converting  the  DoD  acquisition  culture  of  specifications  in  design  and  quality 
standards  to  specifications  of  performance  and  best  commercial  practices.  Federal 
Acquisition  culture  continues  to  evolve  just  as  the  ISO  9000  QMS.  Departure  from  MIL- 
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Q-9858  represents  a  significant  change  for  the  DoD  Acquisition  workforce.  Subsequent 
memorandums  by  Secretaries  of  Defense  Perry  and  Cohen,  as  well  as  the  Under 
Secretary  of  Defense  (USD)  Acquisition  Technology  and  Logistics  (AT&L)  Jacques 
Gansler,  championed  the  adoption  of  commercial  practices.  A  part  of  this  change  was  the 
move  from  MIL-Q-9858,  Quality  Assurance,  and  other  DoD  quality  assurance  guidance 
to  commercial  standards  like  ANSI  9000  and  the  ISO  9000  Series  of  Quality  Assurance 
guidelines.  These  changes  define  and  mark  the  Single  Process  Initiative  introduced  by 
the  Office  of  the  Secretary  of  Defense  in  1995  (23). 

The  move  to  commercial  quality  management  standards  (QMS)  has  not  brought  a 
watershed  of  high  quality  procurements.  Many  of  the  same  weaknesses  that  existed 
under  MIL-Q-9858  remain.  Registered  ISO  9000  organizations  can  and  do  produce 
nonconforming  products.  Inadequate  quality  assurance  programs  exist  under  ISO. 
DCMA’s  role  remains,  as  it  was  prior  to  the  Single  Process  Initiative  and  subsequent 
acceptance  of  commercially  acceptable  quality  management  systems,  to  provide  quality 
audit,  oversight,  and  surveillance.  Discouraging  MIL-Q-9858  has  not  yielded  the 
efficiencies  and  effectiveness  once  thought  possible  under  non- military  standards.  ISO 
9000  is  not  the  panacea  of  QMS  for  the  DoD  or  the  civilian  sector.  One  reason  is  DoD 
procurement  is  not  an  exact  replica  of  civilian  sector  procurement.  DoD  has  numerous 
stakeholders,  often  unable  to  develop  a  requirements  consensus.  DoD  use  of  public  funds 
associated  with  congressional  representation  and  constituents  do  not  reflect  the  bottom- 
line  philosophy  of  corporate  America.  High  risk  is  associated  with  weapon  systems  to 
greater  degree  than  with  private  system  or  product  development.  These  are  a  few  of  the 
factors  bearing  on  difficulties  with  QMS  application  in  DoD. 

The  dynamic  environment  of  business  and  information  management  continues  to 
change  industry  and  DoD  practices.  On  December  15,  2000,  ISO  revised  their  quality 
standard  eliminating  the  five  previous  standards  and  replacing  them  with  two.  The  ISO 
standard  is  followed  by  much  of  the  industrialized  world  and  in  many  aspects  codifies 
international  quality  management  practices  among  industrialized  countries.  While  ISO 
9000  is  a  better  QMS  in  many  ways  and  has  proven  its  worthiness  over  MIL-Q-9858,  the 
acquisition  environment  is  increasingly  complicated  and  complex. 
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Firestone,  a  supplier  of  Ford  for  tires,  represents  a  recent  example  of  ISO 
9000:1994  weaknesses  where  correlations  can  be  drawn  between  the  DoD  and  the 
commercial  sector.  Firestone,  a  QS  9000  registered  corporation,  provided  non- 
conforming  tires  for  the  Ford  Explorer.  The  non- conforming  tires  are  linked  to  numerous 
accidents  of  the  Explorer.  The  link  resulted  in  Explorer  occupant  injury  and  a  lack  of 
public  trust  in  Ford  and  Firestone  products.  The  example  brings  attention  to  one  of  the 
reasons  for  the  new  ISO  9000  standard.  The  ISO  recognized,  as  did  many  of  its  members, 
ISO  9000  must  change  and  adapt  to  some  its  weaknesses.  One  major  company  observed 
that  documenting  one’s  processes  and  ensuring  that  the  documented  processes  are 
followed,  as  required  by  the  ISO  standard,  could  result  in  an  ISO-certified  company 
producing  excellent  but  useless  concrete  lifejackets  (35).  That  particular  company  went 
on  to  develop  its  own  quality  system  that  placed  emphasis  on  continuous  process 
improvement  and  customer  satisfaction,  areas  in  which  many  felt  the  ISO  standard  was 
lacking  (35). 

Has  ISO  9000:2000  addressed  the  weaknesses  of  the  previous  ISO  9000:1994 
version  in  the  dynamic  environment  of  weapon  systems  acquisition?  How  effective  is 
DoD,  specifically  DCMA,  in  properly  evaluating  companies  using  ISO  9000,  and  how 
can  they  become  more  effective  under  the  new  ISO  9000:2000  version?  In  what  ways 
has  and  will  DoD  benefit  from  the  ISO  QMS,  or  not  benefit?  In  short,  what  are  ISO 
9000:2000  effects,  implications,  and  results? 

Modem  quality  management  systems  are  proactive.  The  idea  is  to  design  quality 

and  build  quality  into  the  product  instead  of  reacting  or  inspecting  it  in,  the  previous  DoD 

quality  assurance  focus  (30).  No  single  aspect  in  the  execution  of  services  or 

manufacturing  is  more  important  than  that  of  the  quality  of  the  service  or  item 

manufactured.  The  marketplace  and  DoD  rely  on  the  quality  of  products  and  services 

received.  DoD  quality  managers,  those  responsible  for  quality  management,  and  those 

who  are  beneficiaries  of  quality  hold  great  responsibility  to  the  public  trust,  especially  in 

the  procurement  of  weapon  systems,  due  to  weapon  system  cost,  and  more  importantly, 

to  the  security  of  our  nation  and  the  safety  of  our  military  members.  DoD  knowledge  and 

experience  in  the  area  of  quality  management  systems  (QMS)  protects  that  public  trust.  If 

the  reliance  is  broken,  the  public’s  trust  in  the  product  or  service  is  broken.  The  services 

3 


provider  or  manufacturer’s  quality  systems  are  subject  to  question  when  the  public  trust 
is  broken.  The  object  of  this  research  is  to  explore  the  issues  I  have  outlined. 

B.  OBJECTIVE  AND  RESEARCH  QUESTIONS 

The  scope  will  include:  (1)  a  review  of  commercial  and  Federal  quality 
management  practices,  (2)  an  in-depth  review  of  ISO  9000  and  DoD  quality  management 
systems,  (3)  a  survey  and  evaluation  of  commercial  and  DoD  quality  management 
systems  focusing  on  ISO  9000,  and  (4)  a  study  of  current  DoD  quality  assurance 
practices.  The  thesis  will  conclude  with  recommendations  for  implementing  the  new  ISO 
9000:2000  into  Defense  Contract  Management  Agency  (DCMA)  audits  and  program 
office  evaluation  of  contractors. 

The  primary  research  question  is  what  implications  and  effects  will  ISO 
9000:2000  Quality  Management  System  have  on  the  DoD  acquisition  community? 
Subsidiary  research  questions  include: 

•  What  is  ISO  9000? 

•  What  is  the  Government’s  requirement  for  a  quality  management  system 
and  how  has  it  affected  the  DoD  and  defense  contractors? 

•  How  does  ISO  9000:1994  fit  the  Government’s  quality  requirements  and 
will  ISO  9000:2000  be  a  better  fit? 

•  What  are  the  key  conclusions  and  recommendations  that  might  be  drawn 
regarding  the  ability  relationship  ISO  9000:1994  and  ISO  9000:2000  to 
meet  the  Government’s  quality  requirements? 

C.  METHODOLOGY 

The  methodology  used  in  this  research  will  consist  of  the  following  steps. 

•  Conduct  a  literature  search  of  books,  magazine  articles,  CD-ROM 
systems,  Internet,  library,  and  other  resources 

•  Conduct  a  review  of  commercial  and  Federal  quality  standards  through  a 
review  of  past  and  present  quality  practices 

•  Conduct  a  thorough  review  of  ISO  9000  and  DoD  quality  management 
systems 

•  Examine  the  current  capabilities  and  limitations  of  DoD  and  ISO  9000 
quality  standards 

•  Compare  and  contrast  TQM,  ISO  9000,  and  DoD  quality  management 
systems  addressing  the  advantages  and  disadvantages  of  each 
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•  Evaluate  the  benefits  and  cost  of  ISO  9000  certification,  both  the  1994  and 
2000  versions 

•  Question  both  commercial  and  DoD  practitioners  of  quality  management 
systems 

•  Identify  essential  elements  of  DoD  and  commercial  practices  in  quality 
management  and  how  best  they  can  be  applied  under  DCMA,  and  program 
offices 

•  Identify  areas  of  weakness  and  potential  changes  to  DoD  quality 
management  guidance  in  comparison 

•  Prepare  and  conduct  analysis  of  ISO  9000,  commercial,  and  Federal 
procurement  quality  management  regulation  and  guidance 

•  Recommend  changes  to  current  DoD  practices  in  quality  management 
audit,  evaluation  and  surveillance. 

D.  ORGANIZATION  OF  THE  STUDY 

The  appendices  include  a  comprehensive  glossary  of  abbreviations  and  acronyms, 
a  comparison  to  ISO  9000  versions  2000  and  1994  and  MIL- Q- STD  9858,  and  the 
current  DCMA  Quality  Assurance  Checklist. 

The  research  effort  is  organized  into  five  chapters  and  eight  appendices.  Chapter 
I  presents  the  thesis,  outlining  the  objective,  methodology,  and  organization  of  the 
research  effort.  Chapter  II  highlights  ISO  9000:1994  and  2000  background,  history,  and 
standards.  Chapter  III  presents  and  discusses  the  DoD  quality  management  system 
requirements.  Chapter  IV  provides  perspective  of  quality  assurance  managers  from 
Government  and  DoD  contactors  and  analysis  of  the  implications  and  effects  of  ISO 
9000:1994  and  2000.  I  close  with  recommendations  and  conclusions  in  Chapter  V. 
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n.  iso  9000 


A.  INTRODUCTION 

We  will  begin  the  research  paper  by  providing  some  background  on  quality 
management  systems.  The  background  will  briefly  cover  some  key  guidelines  and 
systems  based  on  performance  and  commercial  standards  from  the  old  MIL- Q- 985 8  and 
Government  defined  standards,  and  provide  an  overview  of  the  ISO:  9000  series  quality 
standards  that  DoD  has  used  as  a  framework  for  evaluating  its  own  suppliers  for  the  past 
seven  years.  The  paper  will  then  follow  with  an  analysis  and  discussion  of  the  salient  and 
pertinent  matters  from  the  issues  above.  The  main  effort  of  the  research  paper  is  to  focus 
on  DoD  management  of  quality  systems  and  their  relationship  with  ISO  9000. 

A  quote  by  Mark  Schaeffer,  Deputy  Director,  Systems  Engineering, 
(DTSE&E/DDSE)  sets  the  tone  and  framework  for  the  thesis. 

Quality  is  not  just  another  interesting  subject,  nor  is  this  just  another 
conference.  Quality  is  one  of  those  overarching  [sic]  considerations  that 
we  all  believe  in — we  all  strive  for — but  that  too  few  seem  to  really 
understand  (48). 

Quality  is  not  a  new  concept  discovered  in  the  ist  century.  Consumers  have 
understood  the  essence  of  quality  since  goods  and  services  were  bought  and  sold  from 
earliest  times.  Quality  is  most  simply  defined  and  recognized  by  the  user,  but  as  there  are 
many  definitions  of  quality,  there  are  many  stakeholders  in  the  effort  of  quality  assurance 
and  management.  Joseph  Juran  defines  the  meaning  of  quality  in  two  areas  of  critical 
importance  (28). 

Quality  means  those  features  of  products,  which  meet  customer  needs  and 
thereby  provide  customer  satisfaction.  In  this  sense,  the  meaning  of 
quality  is  oriented  to  income.  The  purpose  of  such  quality  is  to  provide 
greater  customer  satisfaction  and,  one  hopes,  to  increase  income. 
However,  providing  more  and/or  better  quality  features  usually  requires  an 
investment  and  hence  usually  involves  increases  in  costs.  Higher  quality, 
in  this  sense,  usually  “costs  more.” 

Quality  means  freedom  from  deficiencies  -  freedom  from  errors  that 
require  doing  work  over  again  (rework)  or  that  result  in  field  failures, 
customer  dissatisfaction,  and  customer  claims,  and  so  on.  In  this  sense,  the 
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meaning  of  quality  is  oriented  to  costs,  and  higher  quality  usually  “costs 

less.”  (28) 

Quality,  like  beauty,  has  varied  definitions  and  is  subject  to  the  perspective  of  the 
beholder.  Simply  put  quality  refers  “to  all  the  features  of  a  product  (or  service)  which  are 
required  by  the  customer.”  Quality  management  is  defined  as  what  the  organization  does 
to  ensure  that  its  products  conform  to  the  customer’s  requirements  (7).  Quality 
management  and  assurance  are  explored  in  this  thesis,  focusing  on  Juran’s  second 
definition  of  quality.  DoD  has  traditionally  oriented  procurement  towards  cost. 

Quality  today  is  prevention- oriented  and  process-driven.  This  shifts  the  total 
responsibility  for  quality  from  the  quality  professionals  to  everyone  in  the  organization. 
No  longer  a  "stovepipe"  discipline,  quality  must  be  an  integral  element  of  engineering, 
manufacturing,  and  coding  of  a  product.  Quality  must  be  integral  to  the  way  we  (DoD)  do 
business.  (50) 

B.  HISTORY 

The  US  Military  has  had  a  strong  influence  on  international  quality  standards. 
The  Department  of  Defense  began  developing  quality  standards  in  the  mid-1950s  using 
techniques  and  procedures  used  during  the  industrial  build  up  during  World  War  II  (36). 

The  military  played  an  early  role  in  quality  standards.  As  early  as  1450  B.C.  the 
Egyptians  used  a  form  of  measurements  and  inspection  for  swords.  In  the  Middle  Ages 
craftsmen  both  manufactured  and  inspected  their  work.  Poor  craftsmanship  led  to  no 
business.  Industrialization  separated  the  craftsmen  from  manufacture  and  inspection. 
Honore  Le  Blanc  developed  a  pattern  for  musket  manufacture  for  the  French. 
Manufacture  and  quality  inspection  were  separate  (15).  The  eighteenth  century  created 
continued  specialization. 

In  1798,  Thomas  Jefferson  introduced  one  of  the  earliest  uses  of  U.S.  military 
contracting  and  quality  assurance  when  the  newly  formed  nation  contracted  with  Eli 
Whitney  to  manufacture  10,000  muskets.  The  muskets’  construction  called  for 
interchangeable  parts.  Due  to  problems  associated  with  the  new  technology  of  mass 
produced  weapons  (special  machining,  training  and  inspection)  and  the  Government’s 
discovery  of  frequent  poor  quality,  it  took  Eli  Whitney  nearly  ten  years  to  deliver  the 
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10,000  muskets  originally  contract  for  delivery  in  two  years.  Industrialization  continued 
to  separate  quality  responsibility  from  the  hands  of  the  producer  during  the  early  years  of 
the  U.S.  (15,35) 

During  the  years  of  the  then  War  and  Navy  Departments  and  prior  to  World  War 
Two,  Government  inspectors  practiced  100  percent  inspection  of  manufactured  goods 
and  material.  The  early  view  concerning  quality  was  that  it  was  the  War  Department’s 
responsibility  and  duty  to  assure  and  validate  the  quality  of  the  goods  and  services 
provided  to  the  services.  Assurance  was  gained  through  the  imposition  of  rigid  standards, 
exacting  specifications,  and  the  Government  performing  inspections  throughout  the 
production  process.  (33) 

The  twentieth  century  ushered  new  thought  on  quality  assurance.  Early  pioneers 
of  quality  control  like  Walter  Shewhart,  Howard  Dodge,  George  Edwards,  and  W. 
Edwards  Deming  expanded  thought  and  technology  in  quality  discipline.  Shewhart  led 
his  team  of  Bell  Labs  quality  pioneers  in  the  development  and  use  of  Statistical  Quality 
Control  (SQC).  SQC  went  beyond  mere  inspection  and  the  focus  created  a  method  to 
identify  problems  in  the  process  of  manufacturing  and  production  (15).  World  War  II 
marked  a  shift  to  sampling  methods  as  a  means  of  ensuring  quality.  Due  to  the  large 
amounts  of  material  being  procured  for  the  war  effort  it  became  impractical  and  costly  for 
the  War  and  Navy  Departments  to  continue  reliance  on  100%  inspection  to  assure 
quality. 

The  Government  found  that  it  could  achieve  a  high  degree  of  confidence  in 
product  quality  by  using  sampling  methods.  Statistical  sampling  plans  and  methods  were 
developed  and  used  as  additional  methods  to  assure  quality  while  reducing  Government 
reliance  on  100%  inspection  methods.  (33,35)  This  era  began  the  U.S.  military  use  of 
stringent  standards  on  its  suppliers.  The  War  Production  Board  offered  training  in  the 
new  quality  control  process,  much  like  the  Defense  Acquisition  University  offers  courses 
today  (15).  Dr.  Deming  himself  was  one  of  the  lecturers.  The  aim  was  to  train  people 
how  to  maintain  the  quality  of  delivered  product  while  substantially  reducing  the 
inspection  time.  This  also  meant  a  substantial  reduction  in  the  number  of  Government 
inspectors.  As  a  result,  the  Army  introduced  sampling  inspection  procedures  in  1942  to 
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ensure  and  maintain  quality  without  100%  inspection.  By  1945,  several  similar  plans 
were  available  across  the  Services.  During  the  1950s,  the  DoD  published  three  military 
standards  for  inspection  sampling,  and  in  1963  one  of  these  documents  became  an  ABCA 
(America,  Britain,  Canada,  and  Australia)  Standard.  (50) 

During  the  early  1950s,  as  weapon  systems  continued  to  increase  in  complexity, 
the  Air  Force  pioneered  requirements  for  a  contractor  quality  program.  These 
requirements  were  aimed  at  major  contractors  engaged  in  the  development,  production, 
and  selected  service  tasks  for  aircraft  and  missile  systems.  Following  the  Air  Force 
example,  the  other  Services  initiated  similar  requirements  for  their  products  of  similar 
complexity.  The  OSD  staff  brought  all  parties  together  in  the  late  1950s  to  establish  a 
coordinated  approach  on  quality.  These  organizations  developed  and  issued  Military 
Specification  MIL-Q-9858,  Quality  Program  Requirements.  (50) 

These  standards  are  what  we  currently  call  Military  Quality  Standards  (MIL-Q- 
STD)  (15).  In  the  late  1950’ s  the  DoD  developed  the  MIL-Q-9858  standard  (titled 
“Quality  Program  Requirements”),  which  subsequently  was  used  as  the  model  for  British 
Standard  BS-5750,  released  in  1979  (15,33).  In  the  early  1960’s,  the  DoD  developed  a 
second  standard,  the  MIL- 1-45208  (titled  “Inspection  System  Requirements”). 

One  of  the  models  for  the  ISO  9000  series  of  internationally  accepted  quality 
standards  model  was  BS-5750.  A  great  degree  of  commonality  exists  between  DoD 
MIL-Q  9858  and  the  ISO  9000  series  (15).  The  reader  may  review  Appendix  B  (ISO  to 
MIL-Q  Cross  Reference)  for  a  comparison  of  MIL-Q  9858,  ISO  9000:1994,  and  ISO 
9000:2000. 

MIL-STDs,  however,  were  not  influencing  the  vast  majority  of  commercial 
manufacturing  in  the  post-war  era.  Expanded  commercial  production  called  for  even 
greater  standardization  and  consumer  need  for  higher  levels  of  quality.  The  Japanese 
using,  ironically,  Henry  Ford’s  book,  My  Life  and  Work,  and  heavily  influenced  by 
Deming  developed  the  methods  of  total  quality  management  (TQM)  shifting  quality 
responsibility  back  to  the  worker.  The  U.S.  began  employing  TQM  philosophy  in  the 
early  80s  (15,50).  During  this  time  in  the  late  1980’s  -  early  1990’s  DoD  shifted  to  a 
“management  systems”  approach  to  ensure  quality. 
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The  International  Organization  for  Standardization  (ISO),  founded  in  1947, 
embarked  to  prepare  international  standards  relating  to  quality  assurance  techniques  and 
practices  during  the  international  shift  in  management  philosophy.  A  technical  committee 
was  approved  and  worked  under  the  title  of  Quality  Assurance  ISO/TC  176.  The  ISO 
created  the  world’s  first  internationally  accepted  “generic  quality  management  standards” 
and  outlined  in  the  ISO  9000  family  of  standards  with  the  first  editions  published  in  1987 
(47). 

When  ISO/TC  176  embarked  on  the  development  of  generic  quality  management 
standards  for  worldwide  application,  it  was  able  to  take  advantage  of  a  substantial  base  of 
national  experience,  notably  in  the  United  Kingdom  and  in  Canada.  In  the  United 
Kingdom,  the  BS  5750  standards  were  well  on  their  way  to  broad  acceptance  and,  in 
Canada,  a  series  of  national  standards  known  as  CAN-Z299  were  also  widely  used.  Other 
countries  with  well- developed  quality  management  practices,  such  as  Japan,  also  took  a 
keen  interest  in  the  work  of  the  new  committee.  In  addition,  experience  of  military 
quality  assurance  specifications,  such  as  the  NATO  AQAP  series  (see  Appendix  C 
(NATO  AQAP)),  enriched  the  sources  from  which  TC  176  was  able  to  draw.  (7) 

The  ISO  9000  series  of  International  Quality  Standards  are  an  outgrowth  of 
efforts  by  the  European  Committee  for  Standardization  and  the  ISO.  The  formation  of 
the  European  Union  in  1992  provided  the  major  thrust  in  harmonizing  the  nineteen 
different  European  country  standards  into  one.  The  ISO  9000  series  standards  were 
released  by  the  International  Organization  for  Standardization  in  1987  and  were  updated 
in  1994.  ISO  released  its  newest  version  in  late  2000.  The  International  group  adopted 
the  name  ISO  for  the  standards,  from  the  Greek  word  “iso”  which  means  “equal”  (1,35). 
(The  reader  will  see  the  use  of  ISO  to  be  interchangeable  with  International  Organization 
for  Standardization  (ISO)  as  it  is  used  in  this  paper.)  During  the  latter  part  of  the  1980s, 
DoD  and  many  defense  contractors  discussed  the  possibility  of  using  the  new  ISO  9000- 
ANSI/ASQC  90  series  of  quality  standards  instead  of  military  specifications.  However, 
DoD  did  not  see  any  significant  benefit  in  changing  to  the  1987  version  of  the  ISO  9000 
series. 
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During  this  period  in  the  early  90’s,  a  shift  occurred  in  the  methods  DoD  used  for 
procurement.  Numerous  examples  of  Executive  Branch  and  DoD  policy  and  guidance 
document  the  base  of  our  current  Government  and  DoD  procurement  policy.  A  partial  list 
and  a  brief  synopsis  of  these  examples  are  provided  in  Appendix  E  (Executive  and  DoD 
Policy  Guidance). 

C.  UNDERSTANDING  THE  ISO  QUALITY  SYSTEM 

1.  ISO  Basics 

ISO  9000  is  actually  a  family  of  standards,  referred  to  under  a  generic  title  for 
convenience.  The  family  consists  of  standards  and  guidelines  relating  to  management 
systems,  and  related  supporting  standards  on  terminology  and  specific  tools,  such  as 
auditing  (the  process  of  checking  that  the  management  system  conforms  to  the  standard). 
ISO  9000  is  primarily  concerned  with  "quality  management,"  meaning  what  the 
organization  does  to  ensure  that  its  products  conform  to  the  customer's  requirements.  ISO 
9000  is  also  concerned  with  the  way  an  organization  goes  about  its  work,  and  not  directly 
the  result  of  this  work.  (52,53). 

Nevertheless,  the  way  in  which  the  organization  manages  its  processes  is 
obviously  going  to  affect  its  final  product.  In  the  case  of  ISO  9000,  it  is  going  to  affect 
whether  or  not  everything  has  been  done  to  ensure  that  the  product  meets  the  customer's 
requirements.  However,  ISO  9000  is  not  a  product  standard.  The  management  system 
standards  in  these  families  state  requirements  for  what  the  organization  must  do  to 
manage  processes  influencing  quality  (ISO  9000).  The  philosophy  is  that  these 
requirements  are  generic.  No  matter  what  the  organization  is  or  does,  if  it  wants  to 
establish  a  quality  management  system,  then  such  a  system  has  a  number  of  essential 
features  that  are  spelled  out  in  ISO  (3,52). 

The  ISO  9000  family  of  standards  represents  an  international  consensus  on  good 
management  practices  with  the  aim  of  ensuring  that  the  organization  can  time  and  time 
again  deliver  the  product  or  services  that  meet  the  client's  quality  requirements.  These 
practices  have  been  distilled  into  a  set  of  standardized  requirements  for  a  quality 
management  system,  regardless  of  what  the  organization  does,  its  size,  or  whether  it's  in 
the  private,  or  public  sector.  Full-time  standardization  experts  do  not  develop  ISO 
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standards.  The  business  sectors  most  interested  in  implementing  the  eventual  standards 
are  the  ones  who  provide  experts  to  develop  the  standards.  (3,7) 

The  primary  quality  organizations  representing  U.S.  interests  are  the  following. 
The  American  National  Standards  Institute  (ANSI)  represents  over  250  US  organizations 
that  write  standards,  and  represent  the  U.S.  in  ISO.  The  American  Society  for  Quality 
(ASQ),  formerly  ASQC,  coordinates  with  ISO  and  directs  ISO  activities  in  the  U.S.  on 
behalf  of  ANSI.  The  Registrar  Accreditation  Board  (RAB),  an  ASQ  subsidiary 
corporation,  accredits  quality  systems  registrars  in  the  USA.  The  RAB  also  performs 
initial  evaluations  of  prospective  registrars  and  issue  certificates.  (32,34) 

2.  Generic  Management  System  Standards 

ISO  9000  provides  a  standardized  framework  for  taking  a  systematic  approach  to 
managing  business  processes  (an  organization's  activities)  so  that  they  consistently  turn 
out  product  conforming  to  the  customer's  expectations.  ISO  9000  lays  down  what 
requirements  a  quality  system  must  meet,  but  does  not  dictate  how  they  should  be  met  in 
an  organization  -  which  leaves  great  scope  and  flexibility  for  implementation  in  different 
business  sectors  and  business  cultures. ..as  well  as  different  national  cultures  (52).  This 
flexibility  allows  for  organizations  to  develop  their  own  unique  ways  to  implement  ISO 
standards. 

The  vast  majority  of  ISO  standards  are  highly  specific  to  a  particular  product, 
material,  or  process.  However,  both  ISO  9000  and  ISO  14000  are  known  as  “generic 
management  system  standards.”  Generic  means  that  the  same  standards  can  be  applied  to 
any  organization,  large  or  small,  whatever  its  product  -  including  whether  its  "product"  is 
actually  a  service  -  in  any  sector  of  activity,  and  whether  it  is  a  business  enterprise,  a 
public  administration,  or  a  Government  department  (7,52).  Management  system  refers  to 
what  the  organization  does  to  manage  its  processes,  or  activities.  To  be  really  efficient 
and  effective,  the  organization  can  manage  its  way  of  doing  things  by  systemizing  it.  This 
ensures  that  nothing  important  is  left  out  and  that  everyone  is  clear  about  who  is 
responsible  for  doing  what,  when,  how,  why  and  where  (15). 

The  organization  should  carry  out  auditing  of  its  ISO  9000-based  quality  system 
itself  to  verify  that  it  is  managing  its  processes  effectively  -  or,  to  put  it  another  way,  to 
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check  that  it  is  fully  in  control  of  its  activities.  This  is  a  self-audit.  A  second  party  audit  is 
one  where  an  organization  invites  its  clients  to  audit  the  quality  system  in  order  to  give 
them  confidence  that  the  organization  is  capable  of  delivering  products  or  services  that 
will  meet  their  requirements.  DCMA’s  audits  and  surveillance  are  second  party  audits. 
Organizations  may  engage  the  services  of  an  independent  quality  system  certification 
body  to  obtain  an  ISO  9000  certificate  of  conformity.  This  last  option  has  proved 
extremely  popular  in  the  market  place  because  of  the  perceived  credibility  of  an 
independent  assessment.  It  may  thus  avoid  multiple  audits  by  the  organization's  clients, 
or  reduce  the  frequency  or  duration  of  client  audits.  The  certificate  can  also  serve  as  a 
business  reference  between  the  organization  and  potential  clients,  especially  when 
supplier  and  client  are  new  to  each  other,  or  far  removed  geographically,  as  in  an  export 
context.  (6,52) 

3.  Certification,  Registration  and  Accreditation 

Certification,  registration  and  accreditation  are  often  misunderstood  and 
misleading  and  are  directly  related  to  misconceptions  of  the  ISO  quality  management 
system  and  what  it  can  or  cannot  provide  to  an  organization  and  its  customers  towards 
achieving  a  quality  product  or  service.  In  many  individual’s  minds,  there  is  a  linkage 
between  the  organization  “ISO”  and  the  concept  of  quality  system  certification  and 
registration. 

The  assessment  of  a  quality  system  against  the  requirements  of  one  of  the  ISO 
9000  standards  and  the  subsequent  issuance  of  a  certificate  to  confirm  that  it  is  in 
conformance  with  the  standard's  requirements  is  variously  referred  to  in  different 
countries  as  certification  or  registration.  Likewise,  the  bodies  that  issue  ISO  9000 
certificates  -  “certification  bodies”  -  are  referred  to  in  some  countries  as  "registration 
bodies"  or  "registrars."  In  the  United  States,  “registrars”  is  the  commonly  used  term. 
Again,  these  different  appellations  refer  to  the  same  type  of  body  (7). 

A  final  point  on  terminology  concerns  "ISO  9000  certification."  In  fact,  "ISO 

9000  certification"  means  certification  against  ISO  9001,  ISO  9002  or  ISO  9003  (1994 

version)  or  ISO  9001  and  9004  (2000  version).  The  generic  term  "ISO  9000 

certification"  is  much  more  convenient  than  alternatives  such  as  "ISO  9001/2/3"  or  "ISO 

900x"  certification.  This  is  why  ISO  9000  certification  has  entered  into  common  usage, 
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and  is  employed  in  this  thesis  as  in  other  literary  products  and  common  language. 
However,  only  an  actual  "ISO  9000  certificate"  will  specify  the  standard  against  which 
the  quality  system  in  question  has  been  assessed  and  found  to  be  in  conformance.  (11,52) 

ISO  itself  does  not  carry  out  assessments  to  check  that  users  are  in  conformity 
with  the  requirements  of  the  standards.  Conformity  assessment  -  as  this  process  is  known 
-  is  a  matter  for  suppliers  and  their  clients  in  the  private  sector,  and  of  regulatory  bodies 
when  ISO  standards  have  been  incorporated  into  public  legislation  (7).  The  concept  of 
ISO  9000  certification  and  registration  was  developed  by  the  private  sector.  These 
activities  are  conducted  by  independent  commercial  and  private  organizations,  which 
have  no  direct  affiliation  with  ISO.  (32) 

A  point,  which  logically  follows  from  the  above  remarks,  is  that  it  is  false  to 
describe  a  company  as  "ISO-certified,"  "ISO-registered,"  or  to  use  phrases  such  as  "ISO 
certification,"  "ISO  certificates"  and  "ISO  registration."  ISO  operates  no  system  for 
assessing  the  conformance  of  organizations'  management  systems  with  standards  in  the 
ISO  9000  family.  ISO  itself  does  not  carry  out  ISO  9000  audits  and  awards  no  certificates 
attesting  to  conformity  with  the  standards.  There  is  no  such  thing  as  "ISO  certification," 
or  "ISO  registration,"  whether  in  relation  to  ISO  9000,  ISO  14000,  or  any  other  ISO 
standard.  ISO  9000  auditing  and  certification  are  carried  out  independently  of  ISO  by 
certification  bodies  under  their  own  responsibility.  (7,11) 

D.  ISO  9000:1994 

The  ISO- 9000: 1994  series  consists  of  five  standards,  along  with  some 
complementary  standards,  e.g.,  ISO-8402:  1994  Quality  Management  and  Quality 
Assurance  -  Vocabulary  (4). 

The  five  standards  are: 

•  IS09000-1:  Quality  Management  and  Quality  Assurance  Standards  -  Part 

•  ISO9001:  Quality  Systems  -  Model  for  Quality  Assurance  in  Design, 
Development,  Production  Installation  and  Servicing 

•  IS09002:  Quality  Systems  -  Model  for  Quality  Assurance  in  Production, 
Installation  and  Servicing 

•  IS09003:  Quality  Systems  -  Models  for  Quality  Assurance  in  Final 
Inspection  and  Test 
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•  IS09004:  1994  Quality  Management  and  Quality  System  Elements  -  Part 
1:  Guidelines 

The  ANSI/ASQC  documents  covered  under  ANSI/ASQC  Q-9000  represent 
different  levels  of  quality  requirements  outlined  as  follows:  (4,8) 

•  ANSI/ASQC-Q9001  "Quality  Systems  -  Model  for  Quality  Assurance  in 
Design/Development,  Production,  Installation,  and  Servicing" 

•  ANSI/ASQC-Q9002  "Quality  Systems  -  Model  for  Quality  Assurance  in 
Production  and  Installation" 

•  ANSI/ASQC-Q9003  "Quality  Systems  -  Model  for  Quality  Assurance  in 
Final  Inspection  and  Test" 

•  ANSI/ASQC  Q-9001,  Q-9002  and  Q-9003  are  the  U.S.  equivalents  and 
equal  to  the  international  quality  standards  ISO  9001,  ISO  9002,  and  ISO 
9003,  respectively 


PRODUCT  LIFE- CYCLE 


Figure  1.  ISO  Spheres  of  Influence,  [After  Ref.  35]. 

ISO  9001  is  the  most  far-reaching  standard  of  all  three  Quality  System  Models 
9001,  9002  and  9003,  see  Figure  1  (35).  ISO  9001  covers  a  contractor’s  quality  program 
throughout  its  entire  lifecycle,  from  design,  through  production,  installation  and 
maintenance.  Of  the  three  ISO  9000  documents,  ISO  9001  is  the  most  reflective  of  MIF- 
Q-9858  (41).  ISO  9002  addresses  production,  final  inspection  and  test,  and  installation. 
ISO  9003  covers  only  final  inspection  and  test.  The  following  excerpts  from  ISO-9001 
will  familiarize  the  reader  with  its  twenty  elements.  In  many  cases,  information  from  one 
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element  affects  the  other  elements.  In  this  way,  each  element  influences  the  others.  In  the 
ISO  9000  series  documents  (and  in  many  quality  systems  based  on  this  standard),  the 
word  “supplier”  refers  to  the  company  following  the  quality  standard.  ISO  purports 
managing  suppliers  over  managing  supplies.  (1) 

ISO  9000  provides  a  framework  for  a  quality  system  and  contains  twenty 
elements  as  depicted  below:  (4,35) 


1  Management  Responsibility 

2  Quality  System 

3  Contract  Review 

4  Design  Control 

5  Document  &  Data  Control 

6  Purchasing 

7  Customer  Supplied  Product 

8  Product  Identification  &  Trace  ability 

9  Process  Control 

10  Inspection  &  Testing 


1 1  Control  of  Inspection,  Measuring  &  Test  Equipment 

12  Inspection  &  Test  Status 

13  Control  of  Nonconforming  Product 

14  Corrective  &  Preventive  Action 

15  Handling,  Storage,  Packaging,  Preservation  &  Delivery 

16  Control  of  Quality  Records 

17  Internal  Quality  Audits 

18  Training 

19  Servicing 

20  Statistical  Techniques 


Table  1.  The  20  Elements  of  ISO  9000: 1994,  [After  Refs.  4,35]. 

E.  ISO  9000:2000 

1.  Catalyst  for  Change 

The  major  reasons  for  the  year  2000  revisions  of  the  standards  include: 

•  Emphasizing  the  need  to  monitor  customer  satisfaction 

•  Meeting  the  need  for  more  user-friendly  documents 

•  Assuring  consistency  between  quality  management  system  requirements 
and  guidelines 

•  Promoting  the  use  of  generic  quality  management  principles  by 
organizations,  and  enhancement  of  their  compatibility  with  ISO  14001 

•  Emphasis  on  continual  improvement 

Extensive  surveys  were  performed  by  the  ISO  commission  tasked  with  ISO 
revision,  ISO/TC  176,  on  a  worldwide  basis  to  understand  the  needs  of  all  users  of  the 
quality  management  system  standards.  The  new  revisions  take  into  account  previous 
experience  with  quality  management  system  standards  (1987  and  1994  editions)  and 
emerging  insights  into  generic  management  systems.  ISO  9000:2000  purports  a  closer 
alignment  of  quality  management  systems  with  the  needs  of  organizations  and  better 
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reflects  the  way  organizations  run  their  business  activities.  ISO's  rules  of  procedure  (the 
ISO/IEC  Directives)  also  specify  that  standards  be  periodically  revised  to  ensure  that 
those  standards  are  current  and  satisfy  the  needs  of  the  global  community.  (14) 

ISO  9001:2000  aims  at  guaranteeing  the  effectiveness  (but  not  necessarily  the 
efficiency)  of  the  organization.  For  improved  organizational  efficiency,  however,  the  best 
results  can  be  obtained  by  using  the  new  ISO  9004:2000  in  addition  to  ISO  9001:2000. 
The  guiding  quality  management  principles  of  ISO  9004:2000  are  intended  to  assist  an 
organization  in  ISO  9001:2000  principles  like  continual  improvement,  which  should  lead 
to  efficiency  throughout  the  organization.  (48) 

Efficiency  refers  to  the  “capacity  to  produce  results  with  minimum  expenditure  of 
time,  money,  or  materials.”  Efficiency  focuses  on  the  input- output  ratio.  To  be  efficient 
is  to  do  things  well,  to  attend  to  the  internal  organization  by  refining,  routinizing, 
formalizing,  elaborating  on  existing  knowledge,  and  making  short-run  improvements. 
Effectiveness,  on  the  other  hand,  is  defined  as  “productive  results.”  The  focus  is  on 
doing  the  right  thing  and  an  absolute  level  of  input  or  outcome  determines  efficiency. 
Effectiveness  and  efficiency  comes  from  an  understanding  and  interpretation  of  the 
external  environment  as  it  signals  what  ongoing  adaptations  in  goals,  outputs,  and 
outcomes  are  required.  (49) 

ISO  established  a  3  year  “transition”  period  during  which  accredited  certification 
to  the  1994  standards  and  ISO  9001:2000  may  continue  to  coexist.  This  “transition 
period”  will  end  on  15  December  2003.  By  that  date,  all  organizations  wishing  to  retain 
accredited  certification  will  have  to  bring  their  quality  management  system  to  compliance 
with  ISO  9001:2000.  (52) 

2.  ISO  2000  Family 

The  ISO  9000  family  previously  contained  more  than  20  standards  and 
documents.  This  proliferation  of  standards  was  a  particular  concern  of  ISO  9000  users 
and  customers.  To  respond  to  this  concern,  the  ISO  9000:2000  family  consists  of  four 
primary  standards  supported  by  a  considerably  reduced  number  of  supporting  documents 
(guidance  standards,  brochures,  technical  reports,  and  technical  specifications)  compared 
to  previous  editions.  To  the  extent  possible,  the  key  points  in  the  past  documents  were 
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integrated  into  the  four  primary  standards,  and  sector  needs  addressed  while  maintaining 
the  generic  nature  of  the  standards.  The  four  primary  standards  are  (25): 

•  ISO  9000:  Quality  management  systems  -  Fundamentals  and  vocabulary 

•  ISO  9001:  Quality  management  systems  -  Requirements 

•  ISO  9004:  Quality  management  systems  -  Guidance  for  performance 
improvement 

•  ISO  19011:  Guidelines  on  quality  and/or  environmental  management 
systems  auditing  (to  be  published) 

ISO  9001  and  ISO  9004  create  a  “consistent  pair”  of  quality  standards.  The 
primary  aim  of  the  "consistent  pair"  is  to  relate  modem  quality  management  to  the  actual 
processes  and  activities  of  an  organization,  including  the  promotion  of  continual 
improvement  and  enhancement  of  customer  satisfaction.  (48) 

The  quality  management  system  described  in  the  revised  standard  is  based  on 
quality  management  principles  that  include  the  "process  approach"  and  "customer  focus." 
ISO  9001:2000  and  ISO  9004:2000,  both,  apply  a  process  approach.  The  "20  element" 
structure  of  ISO  9001:1994  has  been  replaced  by  this  process-based  quality  management 
system  (PQM),  which  is  shown  schematically  in  Figure  2,  below.  (28,  48)  For  more 
information  about  the  process  management  approach  and  how  it  is  applied  to  ISO 
9000:2000,  review  Appendix  F  (ISO  Process  Management  Approach). 


19 


CONTINUAL  DrtlROVEMENT  Or 
QUALITY  M AN AGEJil ENT  SYSTEM 


Measure  men}' 

analjeis 

improvement^ 

S  ill *4  idle  i 

Output 

2  r  :-i  lcz 


Customer 
(iso  »*oi)and 

other 

Interested. 

Parties 

(BO  9004) 


Figure  2.  Model  of  a  Process  Based  Quality  Management  System,  [From  Ref.  48]. 

ISO  recommends  the  greatest  value  be  obtained  when  the  entire  family  of 
standards  is  used  in  an  integrated  manner.  ISO  9001:2000  is  the  first  level  of 
performance.  ISO  9004:2000  practices  are  implemented  to  make  the  quality  management 
system  increasingly  effective  in  achieving  business  goals.  ISO  9001:2000  and  ISO 
9004:2000  have  been  formatted  as  a  consistent  pair  of  standards  to  facilitate  their  use  in 
combination  (48).  Although  they  are  "stand  alone"  standards,  they  provide  synergy  when 
used  together,  increasing  organizational  efficiency  and  effectiveness  (25).  The  reader 
may  at  this  time  review  Appendix  D  (ISO  Family),  which  provides  details  of  the  ISO 
9000:2000  Family. 

3.  ISO  2000  Activities  and  Disciplines 

ISO  9001:2000,  like  its  1994  predecessor,  is  used  by  organizations  seeking  to 
establish  a  management  system  that  provides  confidence  in  the  conformance  of  their 
products  to  established  or  specified  requirements.  It  is  now  the  only  standard  in  the  ISO 
9000  family  against  whose  requirements  a  quality  system  can  be  registered  to  by  an 
external  agency  (48). 
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There  are  five  sections  in  the  standard  that  specify  activities,  often  referred  to  as 
the  five  disciplines  of  ISO  9000.  Product  realization  describes  the  activities  used  to 
supply  products;  quality  systems  may  exclude  the  parts  of  the  product  realization  section 
that  are  not  applicable  to  an  organization’s  operations.  The  product  realization  section  is 
the  only  area  where  exclusion  is  an  option.  The  requirements  of  the  other  four  sections 
are:  quality  management  system,  management  responsibility,  resource  management  and 
measurement,  analysis  and  improvement.  These  four  apply  to  all  organizations.  The  five 
sections  of  ISO  9001:2000  define  a  consistent  approach  to  provide  product  that  meets 
customer  requirements.  ISO  9004:2000  is  used  to  extend  the  benefits  obtained  from  ISO 
9001:2000  to  all  stakeholders  (48).  The  ISO  90001  sections  are  organized  into  eight 
standard  clauses.  (26,27)  They  are: 

•  Section  1  -  Scope 

•  Section  2  -  Normative  Reference 

•  Section  3  -  Terms  and  Definitions 

•  Section  4  -  Quality  Management  System 

•  Section  5  -  Management  Responsibility 

•  Section  6  -  Resource  Management 

•  Section  7  -  Product  Realization 

•  Section  8  -  Measurement,  Analysis,  and  ImprovementOf  the  eight  clauses 

of  the  new  standard,  sections  4  through  8  are  the  most  important  to  the  QMS.  These 
clauses  actually  comprise  the  “heart”  of  the  quality  management  system.  Each  one  of 
these  five  main  clauses  has  a  number  of  sub  clauses  which  support  the  concept  identified 
in  the  main  clause.  (32)  The  reader  may  find  additional  perspective  on  the  structure  of 
ISO  9000:2000  by  reviewing  Appendix  G  (ISO  9000:2000  Re-structure). 

4.  Quality  Management  Principles 

Eight  quality  management  principles  provide  the  basis  for  the  revised  ISO 
9000:2000.  Senior  management,  as  a  framework  to  guide  their  organizations  towards 
improved  performance,  can  use  these  principles.  The  principles  contained  in  the  new  ISO 
9000  standards  are  more  “aggressive”  in  nature  than  the  previous  version. 

The  changes  to  the  ISO  standard  series  require  the  user  to  be  more  proactive.  The 
principles  help  to  define  and  validate  the  quality  of  the  work,  goods,  or  services  they 
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provide.  The  principles  concentrate  on  customer  focus,  top-level  management, 
involvement  and  continuous  improvement.  (26,32) 

The  list  below  provides  the  standardized  descriptions  of  the  principles  as  they 
appear  in  ISO  9000:2000  and  ISO  9004:2000.  ISO  states  that  the  goal  and  benefit  of  the 
eight  quality  management  principles,  that  underlie  the  ISO  9000:2000  series,  is  to  form  a 
basis  for  performance  improvement  and  organizational  excellence  beneficial  to  the 
organization’s  quality  management  system.  (26,32) 

•  Principle  1  -  Customer  focus 

Organizations  depend  on  their  customers  and  therefore  should  understand 
current  and  future  customer  needs,  should  meet  customer  requirements 
and  strive  to  exceed  customer  expectations. 

•  Principle  2  --  Leadership 

Leaders  establish  unity  of  purpose  and  direction  of  the  organization.  They 
should  create  and  maintain  the  internal  environment  in  which  people  can 
become  fully  involved  in  achieving  the  organization's  objectives. 

•  Principle  3  -•  Involvement  of  people 

People  at  all  levels  are  the  essence  of  an  organization  and  their  full 
involvement  enables  their  abilities  to  be  used  for  the  organization's 
benefit. 


•  Principle  4  —  Process  approach 

A  desired  result  is  achieved  more  efficiently  when  activities  and  related 
resources  are  managed  as  a  process. 

•  Principle  5  —  System  approach  to  management 

Identifying,  understanding  and  managing  interrelated  processes  as  a 
system  contributes  to  the  organization's  effectiveness  and  efficiency  in 
achieving  its  objectives. 

•  Principle  6  —  Continual  improvement 

Continual  improvement  of  the  organization's  overall  performance  should 
be  a  permanent  objective  of  the  organization. 

•  Principle  7  —  Factual  approach  to  decision  making 

Effective  decisions  are  based  on  the  analysis  of  data  and  information. 

•  Principle  8  —  Mutually  beneficial  supplier  relationships 
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An  organization  and  its  suppliers  are  interdependent  and  a  mutually 
beneficial  relationship  enhances  the  ability  of  both  to  create  value. 

ISO  states  that  the  introduction  of  the  quality  management  principles  is  to  address 
the  proliferation  of  the  20  elements  of  ISO  9000:1994  and  to  better  align  with 
contemporary  philosophy  and  objectives  of  most  quality  award  programs  and  best 
management  practices  (48). 
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HI.  DOD  QUALITY  ASSURANCE  AND  MANAGEMENT  POLICY 

A.  INTRODUCTION 

DoD  QMS  Policy  is  covered  in  a  series  of  Federal  and  Defense  Department 
Regulations,  Policy,  Directives,  and  Guidance.  The  Federal  Acquisition  Regulation 
(FAR),  the  Defense  Federal  Acquisition  Regulation  Supplement  (DFARS),  and  the 
Department  of  Defense  Directive  5000. 2-R  (DODD  5000. 2-R)  encompass  U.S.  DoD 
policy  on  acquisition  and  associated  quality  requirements.  One  of  the  primary  agencies 
charged  with  executing  the  policy  on  quality  management  and  assurance  is  the  Defense 
Contract  Management  Agency  (DCMA).  DCMA  provides  guidance  through  the  use  of 
the  DCMA  Onebook.  DoD  QMS  policy  and  the  current  policy  relationship  to  ISO  9000 
is  based  on  a  series  of  Executive  guidance  documents  published  in  the  1990s,  which  are 
summarized  in  Appendix  E  (Executive  and  DoD  Policy  Guidance). 

The  change  in  DoD  policy  from  military  specifications  to  commercially 
acceptable  quality  standards  resulted  from  concerns  from  industry  that  standards  like 
MIL- Q- 985 8  requiring  significant  documentation  in  areas  such  as  quality  policies  and 
procedures  test  results  and  analysis,  manufacturing  problems  and  proposed  corrective 
actions  were  an  excessive  and  unnecessary  burden  on  industry.  Contractors  and  many 
DoD  employees  suggested  that  much  of  the  documentation  added  little  value  to  the 
manufacturing  and  thus  would  consume  contractor  resources  that  could  otherwise  be 
spent  on  activities  that  add  value  (16).  Many  contractors  were  also  using  dual  QMS: 
MIL- Q- 985 8  and  a  commercial  QMS  such  as  ISO  9000. 

DoD  recognition  of  ISO  9000  pleased  industry  but  concerns  of  DoD  quality 
assurance  auditors  continuing  to  use  MIL-Q-9858  were  high.  Industry  desire  for  a  new 
“higher  level  quality  standard”  policy  addressing  the  shift  in  DoD  procurement  quality 
assurance  began  to  arise.  DoD  addressed  this  issue  by  inactivating  MIL-Q-9858  on  1 
October  1996  and  redesigning  DFARS  246.  In  doing  this  DoD  pushed  contractors  and 
the  Defense  Contract  Management  Command  (now  DCMA)  to  focus  on  commercial 
quality  assurance  standards  and  guidelines.  DCMA  oversight  developed  measures  to 
streamline  existing  oversight  process  to  reduce  contractor  compliance  assets  by  shifting 
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emphasis  from  audits/inspections  to  reviews  of  contractor  processes  (12).  ISO  9000 
allowed  DoD  to  take  better  advantage  of  commercial  best  practices,  lessons  learned  and 
quality  assurance  and  management  techniques  and  procedures.  Contractors  no  longer  had 
to  carry  the  resource  burden  of  managing  two  or  more  quality  programs  or  systems  and 
industry  could  react  more  quickly  to  quality  innovations  and  then  put  them  into  action. 
DoD  realized  two  benefits:  cost  savings  from  DoD  contractors  using  commercial  higher 
quality  standards,  and  the  cost  savings  associated  with  no  longer  maintaining  the  MIL-Q- 
STD  regulations  and  guidelines.  This  chapter  focuses  on  the  DoD  relationship  towards 
ISO  9000. 

B.  DCMA  POLICY 

1.  Contractual  Elements 

Quality  products  and  services  are  fundamental  to  successful  military  operations, 
system  development  and  production.  The  quality  of  products  or  services  is  determined  by 
the  extent  that  they  meet  (or  exceed)  requirements  to  satisfy  customer(s)  at  an  affordable 
cost.  The  Department  of  Defense  (DoD)  acquisition  policy  is  to  emphasize  development 
of  quality  products  through  the  design  of  the  product  and  its  associated  processes  (22). 

The  DCMA  mission  prior  to  award  is  to  assist  in  constructing  effective 
solicitations,  identify  performance  risks,  assist  in  selecting  capable  contractors,  and 
ensure  easily  administered  contracts.  Post  award,  DCMA  ensures  product,  cost,  and 
schedule  compliance,  including  on-site  surveillance  and  program- specific  processes.  (19) 
DoD  contract  quality  requirements  fall  into  four  general  categories  (8): 

•  Contracts  for  commercial  items  (FAR  46.202-1) 

•  Government  reliance  on  inspection  by  contractor  (FAR  46.202-2) 

•  Standard  inspection  requirements  (FAR  46.202-3) 

•  Higher- level  contract  quality  requirements  (FAR  46.202-4) 

Contracting  activities  select  the  appropriate  category  based  upon  the  technical 

requirements,  the  complexity,  the  commercial/Federal  application  of  the  components 
associated  with  the  overall,  and  the  criticality  of  the  associated  system  (22).  When  the 
contracting  activity  determines  a  higher- level  contract  quality  requirement  is  necessary 
and  appropriate,  contractors  may  specify  their  preferred  documented  quality/inspection 
system.  This  system  could  be  based  on  (32,33): 
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More  specifically  FAR  Part  46.401  prescribes  (8): 

Government  contract  quality  assurance  shall  be  performed  at  such  times 
(including  any  stage  of  manufacture  or  performance  of  services)  and 
places  (including  subcontractors'  plants)  as  may  be  necessary  to  determine 
that  the  supplies  or  services  conform  to  contract  requirements.  Quality 
assurance  surveillance  plans  should  be  prepared  in  conjunction  with  the 
preparation  of  the  statement  of  work.  The  plans  should  specify-  all  work 
requiring  surveillance;  and  the  method  of  surveillance. 

Each  contract  shall  designate  the  place  or  places  where  the  Government 
reserves  the  right  to  perform  quality  assurance.  If  the  contract  provides  for 
performance  of  Government  quality  assurance  at  source,  the  place  or 
places  of  performance  may  not  be  changed  without  the  authorization  of 
the  contracting  officer.  If  a  contract  provides  for  delivery  and  acceptance 
at  destination  and  the  Government  inspects  the  supplies  at  a  place  other 
than  destination,  the  supplies  shall  not  ordinarily  be  reinspected  at 
destination,  but  should  be  examined  for  quantity,  damage  in  transit,  and 
possible  substitution  or  fraud.  Government  inspection  shall  be  performed 
by  or  under  the  direction  or  supervision  of  Government  personnel. 
Government  inspection  shall  be  documented  on  an  inspection  or  receiving 
report  form  or  commercial  shipping  document/packing  list,  under  agency 
procedures  (see  Subpart  46.6).  Agencies  may  prescribe  the  use  of 
inspection  approval  or  disapproval  stamps  to  identify  and  control  supplies 
and  material  that  have  been  inspected  for  conformance  with  contract 
quality  requirements. 

•  The  appropriate  International  Organization  for  Standardization  (ISO)  9000 
standard 

•  The  appropriate  American  National  Standards  Institute  (ANSI)  /  American 
Society  for  Quality  (ASQ)  Q9000  standard 

•  The  appropriate  NATO  AQAP  standard  (see  Appendix  C) 

•  A  canceled  Government  standard  with  supplements  (i.e.,  Mil-Q-9858, 
Mil-I-45208) 

•  A  process  control  system  that  meets  other  recognized  industry  (but  non- 
ISO/ANSI/ASQ)  standards 

•  A  process  control  system  that  is  equivalent  to  or  better  than  the  ISO  9000 
standard 

Allowing  contractor  flexibility  to  attain  the  highest  level  of  quality  through 
contractor  selected,  and  subsequently,  DoD  approved  QMS  is  the  goal  of  SPI.  DoD 
encourages  contractors  who  currently  have  contracts  citing  the  MIL-Q-9858A  and  MIL-T- 
45208A  requirements  to  consider  transitioning  into  commercial  quality  system  standards 
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via  the  SPI  (22,23).  The  quality  system  referenced  in  a  contract  is  a  contractual 
requirement  and  the  contractor  is  responsible  for  instituting  and  maintaining  that  quality 
system.  In  most  cases,  the  DCMA  becomes  involved  in  contract  administration  support 
by  request  of  the  procuring  contracting  activity  that  issued  the  contract  (8,22).  It  is  a 
common  misconception  that  DoD  requires  ISO  registration.  Contractors  have  the 
flexibility  to  adopt  any  quality  assurance  and  management  system  so  long  as  it  addresses 
the  customer’s  requirements. 

FAR  Part  46.202-3-90  prescribes  the  insertion  of  a  clause  covering  manufacturing 
process  controls  and  in-process  inspection.  The  clause  introduces  the  possibility  of  ISO 
9000  insertion  in  the  RFP.  The  contractor  then  chooses  ISO  9000  or  other  option.  This 
allows  offerors  to  provide  information  that  will  enable  the  source  selection  team  to  assess 
risk  of  performance,  and  to  identify  those  practices  the  offeror  deems  important  to  the 
project  in  terms  of  the  risk  mitigation.  The  risk  assessment  will  be  based  on  the  ability  of 
the  offeror  to  carry  out  program  requirements  and  evidence  of  demonstrated 
effectiveness. 

2.  Government  Responsibility  for  Produc  t  Quality 

The  Government  bears  the  responsibility  of  determining  whether  the  standard  is 
implemented  properly  and  is  adequate  for  the  risk  involved.  That  determination  usually 
involves  a  quality  system  audit  performed  by  DCMA.  The  Government  does  not  require 
any  contractor  to  become  certified  or  registered  to  ISO  standards.  FAR  Subpart  46.2 
outlines  guidance  concerning  contract  quality  requirements.  Specifically,  46.202-4 
prescribes  higher-level  contract  quality  requirements.  Requiring  compliance  with  higher- 
level  quality  standards  is  appropriate  in  solicitations  and  contracts  for  complex  or  critical 
items  or  when  the  technical  requirements  of  the  contract  require  control  of  work 
operations,  in-process  controls,  and  inspection;  or  attention  to  organization,  planning,  and 
work  instructions.  In  the  RFP  Section  L,  Instructions  to  Offerors  and  Section  M, 
Evaluation  Factors  for  Award,  cover  the  insertion  of  higher- level  quality  system  clauses 
as  outlined  in  FAR  46.311.  The  objective  is  for  the  supplier  to  define  and  mitigate  the 
manufacturing  process  risks  associated  with  the  acquisition  cycle  (4,9). 

Based  on  the  thrust  toward  the  use  of  commercial  standards  and  specifications, 

ISO  9000  has  had  an  ever-increasing  importance  to  Government  contracting.  On  18 
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November  1996,  the  DCMA  (then  DCMC)  issued  Memorandum  No.  96-73,  Quality 
Systems  Evaluation  (Policy),  which  recognized  ISO  9000  and  the  associated  registration 
process.  An  abstract  of  this  letter  follows: 

DCMA/CAO  is  to  perform  ISO  9000  audits  on  selected  contractors  if  the 
contractor  is  moving  toward  implementation  of  ISO  9000.  Audit  results 
are  only  revealed  to  the  customer  (PMO,  Buying  Command)  and  the 
contractor  (35). 

In  support  of  the  use  of  ISO  9000,  DCMA/CAO  will  perform  full  audits 
when  requested  by  the  buying  command  or  if  no  information  is  available 
as  to  the  adequacy  of  a  contractor’s  quality  system  the  CAO  will  verify 
third  party  registration  data,  including  past  audits  and  the  traceability  of 
the  registrar  to  the  Registration  Accreditation  Board  (RAB).  DCMA  will 
accompany  the  third  party  auditors  if  concurred  with  by  the  contractor. 

When  participating  with  an  audit,  the  CAO  auditors  shall  audit 
management  responsibility,  corrective  and  preventative  actions,  and 
internal  audits. 

Upon  successful  completion  of  the  audit,  DCMA  will  issue  a  statement  of 
compliance.  According  to  DCMA,  the  ISO  9000  Series  has  two  primary  roles,  quality 
management  and  quality  assurance.  Two  Types  of  ISO  9000  standards  guidance 
standards  and  contractual  standards  are  currently  in  use.  DCMA  representatives  will  see 
ISO  9000  series  standards  are  used  in  four  situations  (35).  They  are  guidance  for 
Quality  Management,  contractual,  between  first  and  second  parties,  second-party 
approval  or  validation,  and  third-party  certification  or  registration. 

In  most  cases  all  contracts  have  two  “deliverables,”  the  quality  system 
requirements  and  the  goods  or  services  requested.  The  quality  system  requirements  are  as 
much  a  contractual  requirement  as  are  the  goods  or  services  requested.  Validation  of  the 
quality  system  is  as  contractually  necessary  as  validating  the  serviceability  of  the  goods 
or  services.  Quality  Systems  are  a  contractual  requirement.  ISO  9000  fulfills  the 
requirements  of  a  “higher  level”  quality  system. 

DCMA  specialists  must  evaluate  contractor  quality  systems  for  compliance  with 
contractual  higher- level  contract  requirements.  The  ISO  9000  Standards  are  the  most 
widely  used  non- government  standards  in  DoD  contracts.  When  a  “higher  level”  quality 
system  is  called  out  in  a  contract,  that  contract  has  at  least  two  deliverables  -  the  quality 
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system  and  the  material  or  services  requested.  Part  of  the  cost  associated  with  a  contract 
is  the  cost  related  to  the  quality  system.  Just  as  we  verify  the  goods  and/or  services 
contained  in  the  contract  we  also  must  validate  the  contractual  quality  requirement  cited 
in  the  contract.  (32,34) 

The  DCMA  One  Book,  Chapter  4.4,  outlines  how  DCMA  personnel  will  conduct 
contractor  quality  systems  evaluations.  Quality  system  audits  are  tailored  to  examine 
only  those  quality  elements  directed  by  customers,  and/or  those  elements  where  existing 
data  does  not  provide  confidence.  DCMA  personnel  evaluate  the  contractor’s  quality 
system  for  compliance  using  existing  data  from  credible  first,  second,  or  third  party 
audits.  Sample  verification  or  confidence  in  the  auditing  process  may  be  used  to  establish 
the  credibility  of  audits  conducted  by  others.  (8,22) 

Policy  and  guidance  on  the  application  of  quality  standards  is  provided  in  the 
FAR  Part  46;  DFARS  Part  246;  SECDEF  Memorandum  of  June  29,  1994,  subject: 
"Specifications  and  Standards-A  New  Way  of  Doing  Business";  and  USD  (A&T) 
Memorandum  of  December  8,  1995,  subject:  “Single  Process  Initiative"  (13,19,20). 
ANSI/ASQC  Q-9000,  and/or  the  ISO-9000  series  standards  are  referenced  as  model 
quality  management  systems.  Contractors  are  given  the  flexibility  to  respond  with  their 
own  quality  systems.  DODD  5000. 2-R  Part  5  provides  quality  management  guidance  and 
as  the  others  mentioned  above  will  be  referenced  and  further  explored  throughout  the 
discussion  and  analysis. 

DCMA  mission  requirements  have  remained  the  same  and  increased  in  many 
areas  even  though  DCMA  manpower  and  budgetary  resources  have  decreased 
significantly,  from  26,500  in  1989  to  fewer  than  13,000  currently  (19).  Contracting  out, 
or  outsourcing,  increasing  public  scrutiny  of  DoD  procurement  and  budget,  and 
commercial  industry  unaccustomed  to  having  DoD  as  a  customer  have  all  affected  the 
way  in  which  DCMA  conducts  business.  DCMA’s  aging  workforce  has  witnessed  great 
internal  and  external  change  in  DoD  procurement  since  the  late  1980s.  In  order  to 
address  these  issues,  DCMA  uses  risk  planning,  assigning  a  risk  rating  to  suppliers  and 
allocating  resources  to  match  the  risk  associated  with  the  procurement.  Risk  planning, 
assessment,  and  handling  are  directly  correlated  to  supplier  quality  assurance.  The 
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DCMA  One  Book  Section  4.2  addresses  this  area.  DCMA  risk  planning  is  conducted 
through  review  of  contract  and  customer  requirements  and  the  identification  of  quality 
systems,  processes,  and  characteristics.  The  latter  identification  complements  the 
principles  of  the  ISO  QMS.  (45,57) 

DCMA  has  two  tools  at  their  disposal  to  balance  contract  insight  and 
administration  with  the  limited  resources.  The  tools  are  Process  Oriented  Contract 
Administrative  Services,  or  PROCAS  as  it  is  more  commonly  called,  and  Contractor 
Self- Oversight,  CSO.  PROCAS  is  the  DCMA  approach  of  prevention  and  improvement 
rather  than  correction  to  address  and  manage  the  contractor  to  Government  relationship. 
Teaming  and  partnering  between  contractor,  customer,  DCMA,  and  DCAA  are  the 
actions  to  improve  and  analyze  better  quality  management  process.  CSO  is  an  alternative 
to  direct  oversight  by  DCMA.  The  DCMA  and  contractor  sign  a  Memorandum  of 
Agreement  (MOA)  covering  interaction,  schedule  records,  and  changes.  Clearly  defined 
surveillance  plans  must  be  in  place  for  CSO  to  work  effectively.  The  surveillance  plan 
dictates  how  the  CAO  will  monitor  the  contract  CSO.  (54) 

The  DCMA  takes  numerous  audit  approaches  to  guide  their  evaluation  of  a 
contractor  and  these  approaches  are  used  to  discover  underlying  problems  in  a  quality 
system  affecting  areas  of  concern  for  the  ACO  and  PCO.  There  are  varying  indicators 
that  result  in  a  DCMA  Audit  of  a  contractor’s  quality  system:  (8) 

•  Customer  request 

•  Inadequate  contractor  data  to  establish  confidence 

•  Unsatisfactory  performance 

•  Defective  pricing 

•  Internal  control  issues 

•  TQM  program  efficiency 

Quality  system  evaluations  must  be  accomplished  based  on  the  objective  data. 
Sample  verifications  of  key  quality  system  elements  are  authorized  to  validate  the 
credibility  of  reports  of  audits  conducted  by  others.  DCMA  quality  system  audits  focus 
on  the  specific  elements  of  the  quality  system  identified  for  review  by  the  customer,  or 
the  elements(s)  of  the  system  where  confidence  in  compliance  is  lacking.  (8,22,24) 
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Generally,  DCMA  evaluations  and  audits  of  contractor  quality  systems  must  be 
performed  using  (ISO)  or  American  National  Standards  Institute  (ANSI)/American 
Society  for  Quality  (ASQ)  9000  series  quality  system  models.  DCMA  specialists  invite 
customer  participation  in  audits  of  contractor  quality  systems.  The  DCMA  Audit 
Checklist  guides  audit  performance.  A  capable  QMS  is  given  a  Statement  of 
Qualification  by  DCMA.  Whether  the  quality  system  evaluation  results  are  positive  or 
negative,  the  results  must  be  provided  to  the  DCMA  customer(s),  because  they  provide 
valuable  insights  into  the  capabilities  of  a  contractor’s  quality  system.  (8,24) 

C.  DCMA  POLICY  2000 

1.  Policy 

DCMA  Policy  has  not  undergone  any  major  overhaul  since  it  published  Quality 
Management  Systems  -  Requirements  ISO  9001:2000.  The  ISO  standard  meets  the 
criteria  established  by  FAR  Part  46.202-4  as  a  higher- level  quality  assurance 
requirement.  It  is  expected  that  this  standard  will  be  included  in  DoD  contracts  via  FAR 
Clause  52.246-11,  Higher- Level  Contract  Quality  Requirement.  In  accordance  with 
continuing  policy,  the  DoD  does  not  require  accredited  certification  to  higher- level 
quality  standards,  to  include  industry  standards.  DoD  contractors  are  required  to 
maintain  effective  quality  systems  in  accordance  with  contractual  requirements.  DCMA 
issued  Information  Memorandum  No.  01-142  (IM  #  142)  on  February  13,  2001  providing 
guidance  to  the  Commanders  of  Defense  Contract  Management  Districts  (CMD), 
DCMA,  and  Contract  Management  Offices  (CMO).  The  following  paragraphs  focus  on 
DCMA  policy  according  to  IM  #  142  (24). 

Many  defense  contractors  have  based  their  quality  management  systems  on  the 
1994  edition  of  ISO  9001,  9002,  or  9003.  Although  it  is  expected  that  many  defense 
contractors  will  continue  to  base  their  quality  systems  on  the  1994  editions,  DCMA  is 
working  closely  with  the  DoD  and  industry  groups  to  devise  an  approach  for  the 
transition  from  the  1994  editions  of  ISO  9001  to  the  9001:2000  version,  when 
appropriate.  Although  DoD  does  not  require  certification,  the  1994  editions  of  ISO  9001, 
ISO  9002,  ISO  9003  and  ISO  9001:2000  will  co-exist  for  three  years  for  certification 
purposes.  DoD  is  working  toward  a  streamlined  process  on  defense  contracts  to  simplify 
implementation  of  the  ISO  9001:2000  when  elected  by  a  contractor,  provided  no  increase 
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in  price  or  fee  is  required.  The  transition  process  will  incorporate  the  Single  Process 
Initiative  provisions  contained  in  DFARS  211.273.  Transition  activities  will  include  the 
following:  (8,22,24) 

•  Contractors  shall  notify  the  contracting  officer  in  writing  of  this  election  to 
include  their  transition  strategy;  the  use  of  established  Management 
Councils,  including  sector  and  corporate  councils,  is  encouraged  and 
expected 

•  If  DCMA  is  the  cognizant  contract  administration  activity  and  the 
contractor  elects  to  update  their  quality  management  system  from  the  1994 
edition,  DCMA  is  authorized  to  monitor  the  transition,  evaluate 
implementation,  and  approve  and  issue  appropriate  modifications  to 
associated  contracts 

•  If  DCMA  is  not  the  cognizant  activity,  the  cognizant  contracting  officer 
will  generate  appropriate  contract  modifications 

The  ISO  9001:2000  revision  replaces  1994  versions  of  ISO  9001,  9002,  and  9003 
(43).  The  1994  family  of  ISO  9000  documents  originally  comprised  over  27  standards. 

Three  of  the  twenty-seven  were  contractual  standards:  ISO  9001,  ISO  9002,  and 
ISO  9003.  In  the  2000  series  there  will  be  only  one  contractual  standard.  That  standard  is 
ISO  9001,  referred  to  as  ISO  900 1:2000. With  the  release  of  the  new  series,  the  ISO 
9001:2000  is  now  the  only  quality  management  system  standard  in  the  2000  series.  ISO 
9001:2000  allows  the  quality  management  system  to  be  tailored  (22).  The  ISO  9001 
clause,  which  refers  to  this  process,  is  “1.2- Application.”  This  clause  allows  a  level  of 
flexibility  for  the  adoption  of  the  standard  by  various  businesses  (i.e.,  suppliers  who 
would  have  previously  utilized  the  ISO  9002  or  ISO  9003  standard  because  of  the  scope 
of  their  business.)  Clause  1.2  limits  tailoring  to  the  requirements  found  in  clause  7.  The 
quality  manual  must  indicate  if  the  system  is  tailored  and  indicate  the  tailored  clauses  and 
provide  justification  for  their  exclusion.  Tailoring  of  requirements  is  not  a  new  concept  to 
the  DoD.  For  many  years  the  DoD  buying  commands  have  used  tailored  quality  system 
requirements  in  satisfying  their  needs.  Under  the  1994  models,  suppliers  could  choose  the 
quality  system  model,  which  best  represented  the  scope  of  the  work  they  performed  (i.e., 
design  or  development,  production,  etc.).Now  there  will  be  just  one  quality  system 
model,  ISO  9001. Obviously  not  all  suppliers  quality  systems  need  to  embrace  all  the 
clauses  (elements)  contained  in  that  document.  “Tailoring,”  as  it  relates  to  the  revised 
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ISO  9001  standard,  simply  allows  suppliers  to  identify  those  quality  system  clauses 
(elements),  which  apply  to  their  quality  system  (33). 

The  ISO  9001:  2000  allows  suppliers  to  apply  only  those  specific  clauses,  which 
represent  the  scope  of  the  work  that  they  perform.  This  use  of  “application”  however  only 
applies  to  those  requirements  contained  in  clause  7  of  the  ISO  9001:2000  standard, 
Product  Realization.  The  application  can’t  be  used  to  reduce  the  scope  of  suppliers’  QMS 
(8,26,57). 

The  DCMA  One  Book  Supplier  Quality  Assurance,  Chapter  4,  discusses  Quality 
System  Evaluations  and  Audits  in  section  4.6.3. 1.2.  DCMA  specialists  are  not  to  perform 
new  evaluations  or  audits  of  previously  acceptable  ISO  9001  or  9002  contractor  quality 
systems,  due  solely  to  a  transition  from  the  1994  versions  of  the  standards  to  the  ISO 
9001:2000  version.  Specialists  should,  as  necessary,  assess  contractors'  systems 
associated  with  the  differences  between  the  revisions.  Appendix  B  (ISO  to  MIL-Q  Cross 
Reference)  provides  a  comparison  of  ISO  9001:1994  and  ISO  9001:2000.  As  is  the 
normal  DCMA  policy  for  Quality  System  Evaluations,  existing  credible  data  must  be 
used  in  conducting  the  evaluations;  audits  must  be  limited  to  those  clauses  where  existing 
credible  data  does  not  provide  confidence  that  the  system  is  compliant.  Evaluation 
results  may  be  recorded  in  any  convenient  format,  indicating  how  confidence  was 
established  for  each  applicable  quality  system  clause.  DCMA  must  notify  the  contractor 
in  writing  of  the  results  of  this  assessment.  These  written  notices  must  identify  the 
applicable  quality  system  standard  requirements  assessed  (8).  When  assessments  indicate 
significant  noncompliance,  the  written  notice  must  identify  the  exact  areas  of 
noncompliance  and  request  corrective  action. 

2.  Implementation 

The  utilization  of  the  new  ISO  9000  series  standards  impacts  the  DoD  customer, 
the  DCMA  workforce,  and  the  DoD  supplier.  Due  to  the  coexistence  of  the  two  standards 
series  for  the  next  three  years,  greater  attention  will  need  to  be  placed  in  the  identification 
of  the  ISO  9001  revision  referenced  in  contractual  documents  to  ascertain  if  the  ISO 
9001:2000  requirement  and  system  are  tailored.  DCMA  breaks  down  suppliers  into  two 
categories  when  using  ISO  9000  based  systems,  registered/certified  or  non- 

registered/non-certified  (33).  Suppliers  can  continue  to  use  1994-based  standards  and 
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may  be  referenced  in  contracts  as  long  as  it  satisfies  the  customer’s  needs,  although 
registered/certified  DoD  suppliers  with  ISO  9000  based  systems  are  driven  by 
registration  requirements  to  adopt  the  new  standard.  Documents  must  reference  a 
contractual  standard  and  a  specific  version  (1994  or  2000).  If  the  ISO  9001:2000  system 
is  tailored,  DCMA  is  the  activity  assuring  the  tailored  system  will  satisfy  the  DoD 
customer’s  needs.  The  worldwide  acceptance  of  ISO  continues  to  influence 
implementation  of  QMS.  Research  external  to  the  interview  found  that  DCMA  currently 
administers  approximately  24,000  contracts  with  higher- level  quality  requirements 
produced  by  approximately  1,100  suppliers.  Contract  clause  52.246-11  requires  the  use 
of  a  higher- level  quality  requirement,  including  ISO  9001  (20).  Since  the  adoption  by 
DoD  of  ISO  9000  as  allowable  to  use  in  the  early  1990s,  registered  companies  have 
grown  from  225  to  approximately  40,000  companies.  As  of  July  of  2000,  nearly  340,000 
registrations  of  certification  existed  internationally  (15,  25).  As  of  23  May  2000,  ISO 
Certificates  totaled  343,643  in  150  countries.  The  U.S.  has  over  30,000,  Britain  has  over 
60,000,  and  Germany  has  over  30,000.  This  is  an  increase  21%  in  one  year  (54).  A 
survey  of  240  companies  revealed  some  interesting  statistics  about  ISO  9000:1994.  (31) 

•  89  percent  of  certified  companies  and  77  percent  of  non-certified 

companies  agree  that  ISO  9000  is  a  practical  quality  system  for 
manufacturers.  (64  percent  of  certified  and  60  of  non-certified  companies 
say  it  is  practical  for  service  providers.) 

•  73  percent  of  certified  companies  and  67  percent  of  non-certified 

companies  agree  that  ISO  9000  will  save  them  money  in  the  long  run 

•  78  percent  of  certified  companies  and  71  percent  of  non-certified 

companies  say  it  will  “definitely  improve  quality”  in  the  long  run 

•  87  percent  of  certified  companies  and  76  percent  of  non-certified 

companies  say  ISO  9000  is  “necessary  to  remain  competitive” 

•  ISO  9000  a  “value-added  quality  system” 

Since  ISO  does  not  issue  certificates  of  conformity  to  ISO  9000,  there  is  no 
official  ISO  database  of  the  number  of  certificates  or  registrations  issued  by  independent 
auditors.  The  data  is  from  an  ISO-sponsored  survey  of  its  worldwide  membership.  (25) 
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IV.  ISO  9000  FIT  INTO  GOVERNMENT  QUALITY 
MANAGEMENT  REQUIREMENTS 

A.  INTRODUCTION 

How  does  ISO  9000:1994  fit  the  Government’s  Quality  Requirements  and  will 
ISO  9000:2000  be  a  better  fit?  The  analysis  of  this  question  is  based  on  the  data  presented 
in  the  previous  chapters  and  the  appendices.  The  scope  of  this  chapter  considers 
consistent  themes  found  in  the  research.  The  themes  are  defined  and  the  thesis  writer 
presents  an  overall  impression  of  those  themes  to  establish  conclusions  and 
recommendations  for  Chapter  V. 

Appendix  H  Interviews  of  Quality  Managers,  provides  the  raw  interview  data, 
which  serves  as  the  basis  for  this  chapter.  The  interviews  present  a  sampling  of  15 
managers,  directors,  or  auditors  of  quality  assurance  management  departments  from 
companies  that  are  currently  ISO  9000:1994  registered,  and  from  the  DCMA.  Contractor 
organizations  were  selected  based  on  their  prominent  role  in  the  defense  industrial  base. 
DCMA  representation  was  selected  based  on  their  lead  role  in  organizing  and  distributing 
guidance  and  training  on  ISO  9000  and  expert  knowledge  of  military  quality  assurance 
and  management.  The  aggregate  list  of  respondents’  companies  is  presented  in  Appendix 
A  (List  of  Companies  Participating  in  Interviews). 

The  companies  selected  were  manufacturers  of  hardware  and/or  software  for  the 
DoD.  The  scope  of  the  thesis  did  not  include  representatives  from  the  service  industry, 
but  much  of  the  information  drawn  from  this  research  may  apply  to  the  service  industry 
as  well.  All  of  the  respondents  were  involved  in  DoD  contracts  and  are  aware  of  the  ISO 
9000  Standards  and  MIL-Q-9858/MIL-I-45208.  As  a  result,  the  reader  must  recognize 
that  each  respondent  brought  his  own  biases  toward  ISO  9000  standards,  registrars  of 
ISO,  and  the  DoD. 

Of  the  13  private  companies  that  provided  interviewees,  only  one  had  obtained 
ISO  9000:2000  registration.  All  organizations  were  using  an  incremental  approach  to 
move  towards  ISO  9000:2000  registration  directed  from  the  corporate  or  enterprise  level. 
Quality  systems’  planning  was  done  primarily  at  the  corporate  or  enterprise  level  while 
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allowing  respective  business  units  and  departments  to  plan  execution  and 
implementation.  The  incremental  approach  allows  one  business  unit  or  department  to 
take  the  lead  role  in  ISO  9000:2000  implementation.  The  corporation’s  contracted  third 
party  registrar  then  conducts  an  ISO  9001:2000  audit.  The  audit  results  or  lessons- 
learned  are  distributed  throughout  the  corporation  or  enterprise  to  all  other  business  units 
seeking  ISO  registration.  Those  units,  in  turn,  use  the  results  and  lessons -learned  to 
develop  their  own  ISO  transition  plan.  The  incremental  approach  works  with  only  one  or 
two  unit  audits  occurring  simultaneously.  This  allows  subsequent  units  to  seek  and 
acquire  a  higher  level  of  fidelity  into  each  of  their  respective  ISO  implementation  plans. 

Most  of  the  companies  maintain  departments  within  their  organization  or  have 
suppliers  that  retain  MIL-Q  quality  standards  as  their  QMS.  All  of  the  respondents  were 
managers  or  directors  of  quality  assurance  or  management  within  their  organization  and 
all  had  recent  experience  with  ISO  Registrars  and  DCMA.  In  most  cases  the  person 
contacted  was  responsible  for  the  organization’s  transfer  to  ISO  9000:2000  and  was  very 
familiar  with  registrars  of  ISO  and  the  ISO  standards.  Only  one  respondent’s 
organizational  business  unit  decided  not  to  meet  the  ISO  9000:2000  standard. 

This  chapter  consists  of  three  sections,  each  dealing  with  a  theme  emerging  from 
the  interviews.  These  themes  were:  advantages  and  barriers,  policy  vs.  perception,  and 
the  relationships  between  defense  contractors  and  DoD. 

B.  ADVANTAGES  AND  BARRIERS 

The  first  theme  to  emerge  from  the  interviews  was  that  of  advantages  and  barriers. 
This  section  will  draw  from  interview  questions  that  addressed  ISO  9000  registration  and 
use.  Of  the  companies  interviewed,  none  required  ISO  as  a  condition  of  entry  onto  their 
qualified  supplier  list.  However,  ISO,  to  varying  degrees,  is  a  selection  factor.  Most 
defense  contractors  see  ISO  9000  as  advantageous  for  suppliers  to  have  but  fall  short  of 
requiring  ISO  9000  or  third  party  registration.  One  respondent  stated, 

The  imposition  of  the  ISO  registration  process  on  suppliers  could  cause 
financial/technical  barriers  for  them.  To  alleviate  the  problem  we  perform 
audits  to  ensure  that  the  supplier  has  sufficient  control  in-place  to  ensure 
that  product  provided  to  us  meets  or  exceeds  our  customers’  requirements. 
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The  degree  of  confidence  in  the  registration  was  based  primarily  on  two  factors, 
the  confidence  the  prime  contractor  had  in  the  registrar  of  the  potential  supplier  and  the 
past  performance  data  and  reputation  of  the  supplier.  In  all  companies  interviewed,  ISO 
offers  a  reduction  in  the  degree  of  oversight  required  towards  suppliers,  dependent  on 
each  industry  evaluation  of  specific  contract  risk  (59).  A  common  theme  quoted  by 
respondents  is: 

Registered  suppliers  avoid  some  of  the  obstacles  required  to  become  part 
of  approved  supplier  base,  however,  registration  is  not  a  pre-requisite. 

DoD  uses  ISO  registration  in  much  the  same  way  as  commercial  firms.  ISO 
registration  is  a  factor  to  consider  during  risk  management  of  contractor  selection. 
Knowing  that  a  company  follows  ISO,  presumes  that  generally  accepted  baselines  of 
quality  management  and  assurance  practices  are  in  use.  Third  party  registration  increases 
the  level  of  assurance  by  providing  a  level  of  confidence  that  the  quality  system  is 
actually  in  use  and  practiced  by  the  workforce.  Since  registrars  are  independent  of  ISO, 
no  one  registrar  is  identical  to  the  others,  and  therefore  past  performance  and  the  ability 
of  the  supplier  continue  to  play  the  critical  role  in  selection.  In  fact,  many  defense 
contractors  and  suppliers  still  use  MIL-Q-9858  and  MIL-I-45208. 

The  cost  and  benefits  of  ISO  9000  registration  via  a  second  or  third  party  registrar 
must  obviously  be  weighed  by  each  organization.  Worldwide  acceptance  of  ISO  has 
made  it,  either  intentionally  or  unintentionally,  a  requirement  for  entry  into  many 
markets,  especially  overseas  markets.  While  ISO  registration  is  not  mandatory  to 
compete  for  DoD  contracts,  ISO  has  influenced  the  quality  assurance  audit  and 
surveillance  process,  not  only  overseas,  but  in  the  U.S.  as  well.  The  influence  is  reflected 
in  DoD  guidance.  ISO  9000  is  an  example  of  a  standard  that  meets  the  DoD  “higher  level 
quality  standard”  requirement.  Most  organizations  feel  ISO  9000  is  necessary  to  compete 
internationally.  Some  organizations  retain  the  position  that  ISO  9000  is  used  as  a  barrier 
to  market  entry  specifically  in  Europe,  as  stated  by  one  respondent. 

ISO  is  not  value-added  to  our  organization.  It  is  more  of  a  marketing  tool 
than  an  improvement  to  our  quality  system.  Our  registrar’s  advice  and 
surveillance  have  been  of  little  use. 
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It  is  difficult  to  deny  that  ISO  9000  is  a  barrier  to  entry  in  many  markets,  given  its 
worldwide  acceptance.  The  ISO  barrier,  whether  real  or  perceived,  is  based  widely  on 
top  management’s  view  of  ISO’s  value  to  the  organization. 

Many  maintain  that  ISO  9000:1994  was  not  a  significant  change  from  MIL-Q- 
9858  and  that  there  is  little  additional  benefit  or  cost  to  transition  to  ISO  9000:1994.  The 
primary  cost  these  respondents  noted  was  the  high  price  of  third  party  registration.  DoD 
does  not  require  the  third  party  registration  but  many  markets  do,  thus  adding  support  to 
the  view  that  ISO  9000  is  a  marketing  tool.  One  company  position: 

Having  previously  been  a  MIL-Q-9858  company,  the  cost  tradeoff 
between  ISO  registration  activities  (including  internal  audits  and  other 
internal  efforts)  was  equally  counterbalanced  with  the  absence  of  frequent 
ACO/customer  quality  system  evaluations. 

Benefits  ranged  from  increased  productivity,  increased  flexibility,  and  quality 
assurance  discipline,  to  the  absence  or  reduction  of  customer  inspections  provided  as  a 
derivative  of  the  ISO  Third  Party  Registration.  Others  were  more  skeptical,  stating  it  has 
not  brought  a  significant  advantage  to  the  organization  over  MIL-Q-9858  but  resulted  in 
costing  more  for  the  third  party  registration.  Companies  seeing  ISO  as  beneficial  view 
ISO  as  a  catalyst  to  change  the  corporation’s  quality  culture: 

The  most  beneficial  outcome  of  ISO  is  to  have  an  outside  registrar  to 
validate  the  quality  cultural  changes  that  need  to  be  made  in  our 
organization. 

What  we  once  thought  as  unrelated  to  quality  is  actually  related.  It  effects 
the  whole  organization.  Previously  we  looked  at  the  downstream  process, 

ISO  brings  in  the  upstream  processes  that  impact  the  product.  Quality 
goes  to  management  commitment,  [and]  capable  processes  on  the 
manufacturing  floor.  ISO  can  bring  in  the  discipline  to  change. 

To  a  great  degree,  the  actual  impact  of  implementing  ISO  9000  in  an  organization 

depends  on  the  importance  the  organization  places  on  emphasizing  ISO  principles  and 

discipline.  The  standard  is  not  the  panacea  for  quality  problems,  and  without  the  addition 

of  leadership,  management,  and  a  workforce  embracing  a  total  quality  perspective;  ISO 

has  little  chance  to  create  a  recipe  for  success  in  the  organization  implementing  it.  When 

ISO  is  used  as  a  catalyst  for  organizational  change,  incorporating  a  TQM  and  process 

management  approach,  it  can  provide  meaningful  results  in  quality  production,  but  alone, 
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without  a  champion  and  an  organization  to  build  additional  processes  and  systems  around 
ISO  9000,  it  will  merely  meet  the  minimum  requirements. 

More  specifically  many  saw  the  ISO  9000:2000  elements  of  product  realization, 
customer  satisfaction,  and  continual  improvement  as  offering  the  most  significance  and 
concern  by  defense  contractors  and  the  DoD.  Product  realization  is  seen  as  a  mixed 
blessing,  providing  both  gains  and  drawbacks  to  industry  and  the  Government.  The 
concept  allows  for  the  true  commercialization  of  the  QMS;  however,  the  contractor  must 
justify  to  the  Government  exclusions  sought  under  the  product  realization  clause.  Under 
the  Standard  Inspection  Clause,  the  Government  holds  final  approval  of  any  exclusion. 
Excluding  certain  provisions  not  needed  for  contract  quality  assurance  may  provide 
reduced  cost  for  the  Government.  Including  un-necessary  items  and  clauses  under 
product  realization  may  add  unattended  cost  to  a  contract  where  no  direct  benefit  is 
gained  from  inclusion  of  the  clause. 

Respondents  believed  the  2000  version  of  the  ISO  9000  series  addresses 
shortcomings  with  customer  satisfaction  and  continual  improvement,  but  also  commented 
that  customer  satisfaction  and  continual  improvement  introduce  new  problems  and 
concerns.  Concerns  surround  the  abilities  of  DCMA  and  registrars,  or  any  auditor’s 
ability,  to  make  accurate  measures  of  customer  satisfaction  and  continual  improvement  as 
it  applies  to  DoD  contractors.  The  customer  is  not  easily  defined  or  recognizable  in  DoD 
procurement.  Many  stakeholders  are  considered  customers  in  DoD.  DCMA,  DCAA, 
DLA,  Systems  Mangers,  Program  Managers,  Congress,  and  the  users  alike  all  have 
legitimate  claim  to  some  aspect  of  the  DoD  customer.  The  program  office  is  the  user 
representative  to  defense  contractors,  ISO  9000:2000  has  not  changed  that  relationship, 
but  all  stakeholders  assert  some  influence  as  customers  of  the  defense  contractor. 
Registrars  and  DCMA  should  consider  the  unique  situation  in  which  contractors  face  in 
having  DoD  as  a  customer  when  assessing  this  area.  This  is  an  area  that  needs  to  be 
worked  out  as  ISO  9000:2000  is  implemented. 

Continual  improvement  is  another  area  difficult  to  measure  in  many  instances, 
especially  in  mature  products  and  service-based  industry.  For  example,  it  is  easy  to  see 
continual  improvement  in  computers  over  the  past  few  years,  but  is  the  same  true  for 
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laundry  detergent  or  in  the  janitorial  service?  Both  laundry  detergent  and  janitorial 
service  are  examples  of  products  where  it  is  difficult  to  see  noticeable  differences  in 
improvement  unless  one  looks  outside  the  product  itself  for  improvements  in 
management  and  cost  effectiveness.  Again  it  calls  upon  the  auditor  and  PCO/ACO  to 
properly  define  how  these  facets  of  ISO  will  be  addressed  and  measured  in  the  contract. 
In  fact  it  may  be  in  the  Government’s  interest  to  not  evaluate  this  area  at  all. 

Barriers  associated  with  ISO  9000  discussed  by  the  respondents  ranged  from  the 
systemic  to  the  resources  required  in  implementing  the  change.  The  barriers  applied  to 
ISO  9000:1994  as  well  as  implementation  of  ISO  9000:2000.  The  resource  areas  of  time, 
training,  and  financial  cost  topped  the  list.  Systematic  changes  or  challenges,  as  one 
interviewee  said  it,  were  documentation,  awareness  training,  and  the  establishment  of 
systems  to  capture  required  data.  Across  all  industry  interviewees  concerns  were  voiced 
about  respective  registrars  interpretation  of  ISO  9000:2000  and  how  future  audits  will 
vary  and  change  from  the  previous  version  (1994)  audits.  Nearly  all  respondents  again 
mentioned  culture,  which  had  an  overriding  importance  relative  to  other  factors.  If  the 
culture  of  the  organization  cannot  adapt  to  progressive  ways  of  doing  business,  the 
organization  will  cease  to  exist  or  prosper.  The  competitive  forces  of  the  market  place 
will  replace  organizations  that  cannot  adapt,  evolve  and  change  to  meet  ever  increasing 
and  technologically  challenging  customer  requirements.  The  general  perspective  and 
consolidation  of  interview  comments  by  industry  and  DoD  is  presented  below: 

The  initial  barriers  or  challenges  to  ISO  9000:2000  certification  are: 

•  An  initial  and  significant  effort  in  mapping  or  restructuring  one’s 
documentation  system  to  align  with  the  standard. 

•  An  initial  increase  in  the  need  for  employee  awareness  training  due  to 
additional  requirements 

•  Establishing  the  budget  for  training  internal  auditors  and  hiring  a  registrar. 

•  Designing  a  system  to  capture  evidence  of  improvement  programs  and 
upper  management’s  involvement. 

•  Re-calibration  between  [company]  and  [their]  registrar  regarding  some  of 
the  new  requirements  and  exactly  what  they  mean.  (Based  on  the  1987  to 
1994  transition,  the  variation  of  interpretation  reduces  with  time;  but  the 
initial  differences  in  opinion  starts  out  fairly  wide). 
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•  The  culture  of  “us  versus  them,”  between  the  quality  inspectors  and 
production,  is  difficult  to  change.  The  culture  change  needed  is  the  move 
to  make  everyone  responsible  for  quality  assurance  in  the  organization, 
resulting  in  a  better  product.  Making  each  one  focus  on  their  [sic] 
processes  in  quality  management  so  [they]  do  not  have  the  performance 
and  defect  problems  to  begin  with.  The  goal  is  to  make  sure  all  the  various 
partners  in  [the]  company  take  responsibility  and  accept  their  function  for 
the  quality  of  what  they  do  whether  it  is  a  process  or  a  product.  This  helps 
every  one  understand  that  quality  is  everyone’s  business. 

C.  POLICY  VERSUS  PERCEPTION 

A  large  degree  of  misconception  and  misunderstanding  is  apparent  between  the 
policy  of  DoD  QMS  and  ISO  9000  and  the  perceptions  of  both  the  DoD  and  defense 
contractors.  This  section  is  primarily  based  on  interviews  (see  Appendix  H),  Quality 
Process  Versus  Quality  Product  and  examines  areas  where  disconnects  exist  between 
policy  reality  and  policy  perception.  This  section  provides  insight  and  understanding  into 
that  policy. 

Industry  respondents  were  at  varying  stages  in  the  ISO  9000:2000  transition 
process.  Opinion  within  organizations  varied  in  the  degree  to  which  the  new  ISO  might 
effect  the  organization.  Those  who  thought  the  new  version  would  change  the 
organization’s  QMS  to  a  g'cat  degree  seemed  to  be  placing  their  hopes  on  continuing  to 
shift  organizational  quality  culture.  Respondents  saw  the  external  force  of  ISO 
registration  as  a  catalyst  to  make  more  changes  to  internal  quality  systems  and  move  the 
organization  toward  more  process  management  to  enhance  the  quality  of  manufacturing. 
Dissenting  opinion  was  that  ISO  changed  little  and  that  planning  was  focused  on 
updating  documentation  requirements  as  published  by  respective  registrars  to  meet  the 
registration  to  ISO.  All  organizations  that  participated  in  interviews  were  using  an 
incremental  approach  to  registration,  viewed  from  the  corporate  or  enterprise  level. 

Some  see  ISO  9000  registration  as  a  marketing  tool  rather  than  a  quality 
assurance  tool  or  standard  valuable  to  the  organization,  as  mentioned  in  section  B  of  this 
chapter.  To  some  extent,  it  may  be  true.  ISO  Registrars  are  independent  of  ISO  and  as 
with  any  independent  organization  are  subject  to  the  bottom  line  of  profit  in  order  to 
survive.  A  case  could  be  made  that  ISO  is  used  as  a  barrier  to  entry  in  the  market  place; 
especially  in  Europe  where  ISO  is  more  predominate.  ISO  9000  certification  is  both  a 
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marketing  tool  and  a  good  quality  management  system.  ISO  9000  provides  no 
competitive  advantage  due  to  its  worldwide  proliferation.  It  is  best  described  as  a  quality 
system  baseline  within  industry  and  provides  a  common  international  version  of 
generally  accepted  quality  management  standards  and  assurance  techniques  to  provide 
goods  or  services  in  the  global  market  place. 

As  the  reader  will  see  in  a  later  section  on  quality  process  versus  quality  product, 
meeting  a  process  can  still  lead  to  manufacturing  scrap.  The  process  must  match  a  set  of 
industry-  and  organization- specific  set  of  metrics  that  measure  the  effectiveness  and 
efficiency  of  processes  that  produce  an  item  meeting  the  conformity  of  the  customer’s 
requirements.  Product  specific  measures,  organizational  leadership,  workforce 
experience,  culture,  and  training  all  play  a  role  in  total  quality.  No  standard  can  dictate 
entrepreneurial  uniqueness  to  guide  a  manufacturer  to  it  market  niche.  Two  opposing 
views  of  ISO  9000:2000  are  presented: 

First,  the  “barrier  to  market  entry”  view: 

What  was  unique  about  ISO  was  the  registration  process.  It  was  used  as  a 
barrier  for  entry  into  the  common  market.  The  registration  framework 
makes  it  successful  and  perpetuates  the  success,  financially,  of  the 
companies  that  are  registrars.  It  is  a  product  or  service  the  registrars 
provide  for  sale.  ISO  is  a  commercial  product,  not  a  significant  customer 
support. 

Second,  the  view  that  ISO  9000  makes  a  difference: 

I  see  the  ISO  changes  as  significant.  A  heavy  emphasis  on  continuous 
improvement,  management  oversight,  record  meeting.  It  places  more  a 
burden  on  contractors  to  document  their  continuous  process  improvement. 

I  disagree  that  the  five  disciplines  talked  to  in  the  new  ISO  were  already 
built  into  the  previous  20  elements. 

The  Firestone  example,  introduced  previously,  brings  attention  to  one  of  the 
reasons  for  the  new  ISO  9000  standard.  ISO  recognized,  as  did  many  of  its  members,  that 
ISO  9000:1994  must  change  and  address  some  of  its  weaknesses.  One  major  company 
observed  that  documenting  one’s  processes  and  ensuring  that  the  documented  processes 
are  followed,  as  required  by  the  ISO  standard,  could  result  in  an  ISO-certified  company 
producing  excellent  but  useless  concrete  lifejackets  (35).  That  company  went  on  to 
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develop  a  quality  system  that  emphasized  continuous  process  improvement  and  customer 
satisfaction,  areas  in  which  many  felt  the  ISO  9000:1994  standard  were  lacking  (35). 

The  eight  QMS  Principles  of  ISO  are  a  solution  intended  to  address  problems 
with  ISO  9000:1994:  such  as  demonstrated  processes,  evidence  of  product  conformance, 
and  documentation  as  well  as  customer  satisfaction  and  continual  improvement  (14,35). 
The  new  standard  places  more  emphasis  on  management’s  involvement  in  quality 
systems  and  process  and  requires  auditors  to  spend  more  time  and  effort  verifying 
effectiveness  through  interviews  of  people  on  the  production  floor  or  conducting  the 
service.  The  shift  of  ISO  9000  from  the  20  elements  to  the  eight  management  principles 
provides  an  excellent  framework  for  teaming  and  implementing  the  principles  in  an 
integrated  approach  rather  than  a  stovepipe  one.  Many  organizations  had  divided  the  20 
elements  into  respective  departments  to  administer,  creating  a  stovepipe  approach  to  ISO 
9000:1994  implementation.  The  eight  principles,  which  embody  the  20  elements,  cause 
organizations  to  take  an  integrated  approach  to  their  performance  and  execution.  Some 
defense  contractors  perceive  the  20-element  elimination  as  a  needed  improvement. 

The  only  improvement  that  I  see  in  the  2000  version  is  that  it  is 
restructured  to  eliminate  the  perception  that  ISO  9000  is  20  separate,  stand 
alone  elements  that  “the  quality  guy”  is  responsible  to  ensure  are  met. 
Unfortunately,  a  large  degree  of  vagueness  in  the  requirements  of  the 
standard  accompanies  the  new  revision. 

Others  see  the  principles  and  disciplines  of  ISO  9000:2000  as  intrinsic  properties 
of  the  20- elements. 

What  is  obvious  to  some  in  the  previous  20  elements  is  not  so  to  others. 

What  organizations  see  as  shortcomings  or  weaknesses  and  even  strengths 
of  ISO  is  largely  dependent  on  the  culture  of  the  organization 
implementing  ISO. 

No  matter  the  opinion,  organizations  that  adopt  ISO  9001:2000  will  have  to 
demonstrate  that  their  product  realization  processes  are  effective  and  producing  quality 
product.  They  will  also  have  to  demonstrate  that  product  conformity  or  critical 
characteristics  are  acceptable  (35),  and  must  establish  plans  for  improvement. 
Improvements  must  then  be  actually  demonstrated.  Contractual  acquisitions  for 
commercial  and  non- complex  items  would  not  normally  require  higher- level  quality 
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assurance  requirements  such  as  ISO  9001:2000.  If  the  requirements  of  the  quality 
management  system  are  being  fully  met,  then,  processes  are  in  control,  product 
conformity  levels  are  at  acceptable  levels,  and  substantial  plans  for  continual 
improvements  are  in  place.  (4,14,35) 

Interviews  and  thesis  research  reflect  that  ISO  9000  is  more  effective  when  used 
with  advanced  quality  procedures.  Quality  product  development  involves  a  broad  range 
of  quality  techniques  and  measurement.  Numerous  inspection,  modeling,  and  sampling 
techniques  are  in  use  in  addition  to  ISO  9000  by  the  defense  contractors  interviewed. 
The  quality  system  in  use,  as  one  respondent  put  it,  “  must  be  flexible  and  dynamic... 
with  a  list  of  menu  options  or  courses  of  action  to  follow  to  implement...  in  order  to 
distinguish  between  a  trend  and  an  isolated  incident  (59).” 

One  quality  manager  with  an  extensive  software  program  more  specifically  made 
the  following  remark, 

We  have  many  pre-ship  quality  processes.  Most  are  based  on  the  CMM 
Model  [sic]  and  follow  CMMI  and  internal  standard  processes  [The 
respondent  was  referring  to  the  Capability  Maturity  Model  (CMM),  and 
the  associated  Capability  Maturity  Model  Integration  (CMMI)].1  We  have 
a  process  that  looks  at  and  measures  the  critical  paths  of  a  product;  design, 
build,  manufacturing,  test  and  function,  focusing  on  the  products’  “critical 
characteristics.”  We  also  use  break  and  fix,  accelerated  life  testing  to 
mature  our  products  to  measure  quality  level. 

On  the  production  and  service  side,  we  have  numerous  management 
reviews,  various  avenues  for  customer  feedback,  tracking  systems  like 
systems  defect  trends.  All  of  these  procedures  are  linked  to  our 
organization’s  failure  review  board  involving  all  product  players 
(stakeholders).  Suppliers  and  customers  are  a  part  of  the  entire  process 
and  manufacturing  relationship.  An  output  is  our  Quality  System  Report 
Card  providing  our  process  to  improve. 

A  clear  distinction  is  made  between  an  organization  with  the  ability  to  produce  a 
quality  product  and  one  that  actually  does  produce  a  quality  product.  An  organization 
can  meet  registration  requirements,  proving  it  has  a  system  of  quality,  but  can  and  may 


'The  respondent  was  referring  to  the  Capability  Maturity  Model  (CMM),  and  the  associated  Capability 
Maturity  Model  Integration  (CMMI).  The  Software  Engineering  Institute  at  Carnegie -Mellon  University 
developed  CMM  and  CMMI.  DoD  5000. 2-R  requires  software -intensive  Major  Defense  Acquisition 
Programs  (MDAPs)  to  meet  certain  requirements  specified  in  the  CMM. 
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still  produce  material  that  is  nonconforming  or  even  defective.  A  popular  analogy  can  be 
made  between  a  surgical  operation  and  a  quality  system  and  how  measures  of  the  system 
or  process  may  not  be  appropriate. 

Hospitals  are  expected  to  produce  quality  but  performance  based  on 
quality  alone  is  never  measured,  in  part  because  the  meaning  of  the  term 
quality  is  both  elusive  and  disputed  [as  the  researcher  has  noted  in  Chapter 
II].  For  instance,  physician  claims  that  quality  stems  from  using  the 
“correct  procedure,”  no  matter  what  the  outcome  (41). 

A  surgical  procedure  or  process  may  be  high  quality  but  if  it  does  rot  match  the 
end  result  of  increasing  the  health  and  well-being  of  the  patient,  is  it  successful?  The 
answer  to  the  surgeon  may  possibly  be  yes,  but  to  the  patient  of  a  surgical  procedure  the 
answer  is  no. 

The  Government  is  not  required  to  accept  a  QMS  until  it  demonstrates  that  it 
satisfactorily  controls  process  and  product  in  accordance  with  the  Standard  Inspection 
Clause.  QMS  demonstrations  and  audits  may  not  discover  faults  in  QMS  until  contract 
execution.  Government  auditors,  while  afforded  protections  outside  the  scope  of  ISO 
9000,  must  continue  to  be  diligent  in  audit  and  surveillance  procedures  even  under  ISO 
9000:2000. 

Elaborate  systems  of  checking  to  ensure  procedural  fidelity  are  administered  but 
may  not  meet  the  results  or  desired  outcome  of  the  customer  (41).  Organizations  that  are 
realizing  products  must  have  the  institutional  ability  and  discipline  to  implement  the 
underlying  essence  and  concepts  of  the  ISO  9000  Standard,  not  merely  the  “shall”  and 
“will”  of  the  document.  In  this  area  ISO  9000:2000  is  addressing  a  major  deficiency  that 
has  been  a  source  of  international  criticism.  ISO  9000:2000  yields  private  sector 
protections  not  previously  provided,  and  makes  ISO  9000:2000  congruent,  in  many 
aspects,  with  the  protections  of  the  Government’s  Standard  Inspection  Clause. 

The  ability  to  become  a  registered  ISO  organization  and  still  produce  scrap, 
especially  under  the  ISO  9000:1994  version  was  evident  industry-wide.  ISO  certification 
means  only  that  the  company’s  affirmation  is  that  it  does  what  is  says  it  does.  The 
outcome  could  be  good  or  bad,  as  seen  in  the  hospital  analogy.  The  ISO  effort  to  address 
this  shortfall  is  evident  in  the  eight  management  principles  of  ISO  9000:2000, 


47 


particularly  customer  satisfaction,  continual  improvement,  and  process  management 
approach.  The  eight- management  principles  and  five  “disciplines”  of  ISO  9000:2000  are 
too  new  to  develop  a  statistically  sound  judgment  if  the  principles  and  disciplines  provide 
a  higher- level  of  quality  assurance.  Often  only  one  degree  of  separation  exists  between 
ISO  9000  registered  companies,  which  consistently  produce  conforming,  defect  free 
material  and  those  who  are  registered  and  do  not.  A  respondent  acknowledged: 

No  registration  process  will  guarantee  product  quality.  The  guarantee 
audit  would  be  so  expensive  that  it  would  be  unaffordable,  in  addition, 
there  are  very  few  people  that  could  perform  an  audit  to  the  scope  and 
detail  required  to  verify  every  step  in  every  process  is  acceptable.  Even 
100  percent  inspection  is  only  85%  effective. 

Another  respondent  stated:  ISO  registration  indicates  that  you  have  the 
minimum  necessary  business/management  processes  in  place;  it  is  the 
starting  point  for  stellar  quality  not  the  zenith. 

The  only  absolute  measure  of  conformance  is  100%  inspection  and  as  was 
realized  during  the  early  stages  of  WW  II,  to  conduct  100%  inspection  is  cost  prohibitive 
in  manufacturing  mass  quantities  of  goods  or  technically  difficult  items  to  test,  like 
software  and  modem  weapon  systems.  Measures  such  as  random  sampling  and  statistical 
process  control  must  be  established  as  part  of  an  organization’s  metrics  to  measure 
product  conformance.  ISO  9000:2000  provides  the  requirement  for  product  quality 
assurance  but  there  are  intangibles:  warranty,  consistency,  quest  for  quality,  process 
tools  to  continually  improve,  cost  analysis,  and  product  process  quality  issues  and 
resolution  (59). 

Another  point  to  consider  is  that  not  all  registrars  or  DCMA  auditors  are  created 
equal,  some  are  more  conscientious  than  others,  and  each  comes  with  his  or  her  own 
personal  biases  and  backgrounds  influencing  the  audit  (9).  Just  as  not  all  of  the  operators 
of  a  piece  of  equipment  in  a  factory  are  equally  trained  or  equally  proficient,  neither  are 
registrars  and  DCMA  auditors.  In  the  researcher’s  view,  quality  assurance  and 
management  requires  an  integrated  product  and  process  development  (IPPD)  approach  of 
teaming  between  industry,  DCMA,  and  registrars  to  provide  the  greatest  level  of  fidelity 
to  build  a  quality  product  through  the  systems  engineering  approach  and  IPPD.  The  IPPD 
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and  Systems  Engineering  approach  fit  well  into  the  ISO  quality  management  principles  in 
addressing  continuous  process  improvement  and  PQM.  As  one  interviewee  responded: 

[You  need  to]  understand  the  essence  of  the  processes  and  that  is  what 
Process  Based  Management  helps  you  to  do.  The  fact  that  everyone  is 
looking  at  their  processes  through  established  metrics,  and  measuring  the 
process  against  the  metrics,  you  would  be  hard  pressed  to  have  a  major 
systems  failure. 

You  can  fool  the  auditors  but  if  you  use  them  as  a  tool  to  improve  your 
process  you  can  use  that  information  to  make  real  improvements  with  your 
company.  People  within  the  organization  must  embrace  the  need  for  their 
role  in  the  quality  process  of  the  whole  organization  and  its  product. 

DCMA  plays  a  critical  role  in  the  policy  versus  perception  of  a  quality  system,  or 
process  versus  the  desired  outcome  of  quality  conforming  product.  DCMA’s 
involvement  as  contract  administrator  and  evaluator  of  organizational  quality  systems 
during  solicitation  and  during  daily  operations  is  of  critical  importance  in  the  dynamic 
environment  of  quality  assurance  in  DoD  procurements.  Experts  agree  no  one  standard 
or  preaward  survey  can  provide  absolute  guarantees  of  a  100%  defect  free  product. 
Standards  and  surveys  can  and  do  provide  valuable  insight  into  an  organizations  ability  to 
produce  a  quality  product. 

The  key  to  creating  a  quality  product  is  to  find  the  congruence  between  quality 
systems,  manufacturing  processes,  and  conforming  product.  Once  congruence  is  found, 
then  an  organization  must  establish  metrics  to  continually  monitor  the  process  and  allow 
feedback  at  all  levels  to  continually  improve  and  address  quality  issues  associated  with 
not  only  the  product  but  also  the  processes  used  to  manufacture  the  product.  When 
congruence  is  broken,  the  source  of  failure  and  the  process  that  created  the  fault  must  be 
identified  and  explored.  Corrective  action  taken  then  is  distributed  to  all  the  stakeholders 
in  the  process  that  created  the  fault.  The  distribution  of  corrective  action  allows  for 
continual  process  improvement  of  the  organization. 

The  goal  of  any  quality  policy  is  to  address  quality  risk,  as  one  major  defense 
contractor  summarized: 

Quality  addresses  risk.  Well-defined,  well- implemented  quality  systems 
greatly  increase  the  percent  of  time  you  have  no  defects. 


49 


Varying  opinions  were  found  on  what  type  of  risk  ISO  9000  addressed.  Critical 
opinions  of  the  change  saw  the  ISO  9000  update,  as  a  move  to  focus  more  on  supplier 
needs  than  those  of  the  customer.  Critics  suggest  that  the  ISO  standard  was  “watered 
down”  to  allow  registrars  more  latitude  in  ISO  9000  registration,  thus  increasing  the 
marketability  and  profits  of  registrars  of  ISO  and  registrars  customers.  The  elimination 
of  the  20  elements  in  their  opinion  allows  for  broader  and  greater  interpretation  of  the 
ISO  standard. 

Opponents  see  a  gradual  trend  among  registrars  favoring  suppliers  rather  than 
buyers.  Opponents  believe  the  ISO  focus  is  on  producer  risk  rather  than  consumer  risk. 
The  focus  creates  an  environment  allowing  registrars  of  ISO  to  certify  more  companies 
and  thus  increase  the  registrar  market  share  and  role  in  ISO.  The  researcher  could  find  no 
evidence  to  support  or  deny  this  claim.  However,  to  some  degree  this  dilemma  always 
occurs  in  manufacturing.  Manufacturers  balance  producer  versus  consumer  risk  daily 
and  manage  the  tradeoffs  the  two  risks  hold.  Industry  observations  espousing  ISO  focus 
on  producer  risk  followed  these  comments. 

According  to  one  respondent: 

The  buyer  community  has  not  been  addressed  in  the  2000  version.  The 
customers  of  the  registrars  influenced  ISO  2000.  I  see  it  as  a  less  effective 
document  and  aimed  more  towards  the  service  industry  rather  than 
manufacturing. 

According  to  another: 

The  new  version  is  more  concerned  with  producer  risk  than  consumer  risk. 

The  ISO  standard  is  neither  good  or  bad,  it  is  not  a  silver  bullet  of  quality 
to  an  organization  either.  I  see  three  new  areas  in  ISO  2K,  continuous 
process  improvement,  quality  objectives,  and  customer  satisfaction.  While 
these  are  new  to  the  document  does  it  help  be  make  better  widgets?  The 
whole  focus  of  the  three  is  nothing  new  [and]  any  good  business  is  doing 
these  things  already  or  they  would  not  be  or  continue  to  be  in  business. 

The  document  should  be  intuitive  for  any  competitive  market  in  a 
capitalistic  society.  ISO  provides  no  edge  it  is  generic  [;]  it  wants 
everyone  to  be  gray. 

Proponents  of  the  change  believe  ISO  better  matches  business  movement  to  a 
process  management  approach.  The  same  approach  is  used  in  state-of-the-art  quality 
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assurance  systems.  The  new  standard  also  better  resembles  internationally  recognized 
quality  awards  programs.  The  new  standard  creates  a  better  framework  from  which  an 
organization  can  build  and  define  its  own  QMS. 

Often  what  organizations  saw  as  either  strengths  or  weaknesses  of  the  previous 
versions  of  ISO  9000  was  based  on  their  own  point  of  reference  and  ability  to  understand 
what  some  call  the  implied  tasks  of  ISO  9000:1994.  Interviewees  saw  the  change  as 
addressing  many  of  the  implied  and  not  so  well  defined  aspects  of  the  1994  version.  The 
framework  of  ISO  9000:2000  is  consistent  with  the  total  quality  management  concept  and 
DoD  5000.2-R.  The  common  goal  in  managing  an  organization  is  to  create  an 
environment  so  that  it  excels  in  all  dimensions  of  products  and  services  that  are  important 
to  the  customer  (4).  The  following  table  will  visually  show  the  consistency  between  the 
three  with  each  area  outlining  their  respective  tenets. 


TQM 

ISO  9000:2000 

DoD  5000.2-R 

Committed  Involved  Mgmt 

Responsible  Management 

Monitor  Critical  Processes 

Customer  Focus 

An  intrinsic  ISO  goal 

Customer  Feedback 

Workforce  empowered 

Internal  Quality  Audits  /  Mgt 
Principle  (c.)  Involvement 
of  people 

Proactive  not  Reactive 

Continuous  Improvement 

Corrective  Action  &  Process 
Control  /  Mgt  Principle  (f) 
Continual  improvement 

Continuous  Process 
Improvement 

Partnering  w/  Suppliers 

Intrinsic  Control  of  Product 
Goals 

Root  Cause  Analysis  & 
Corrective  Action  Systems 

Performance  Measures 

Statistical  Techniques 

Capable  Processes 

Table  2.  TQM,  ISO  9000,  and  DoD  5000.2-R  Relationship,  [After  Refs.  8,26,47] . 


The  respective  tenets  of  TQM,  ISO  9000,  and  DoD  5000.2-R  serve  a  common 
purpose.  They  all  support  a  better  quality  environment.  DoD  5000.2-R  is  a  DoD 
Regulation  providing  guidance  and  implementation  of  the  DoD  Acquisition  System, 
specifically  DoD  5000-2R  (Interim),  Part  5.2.3.  ISO  9000:2000  is  an  international 
standard  or  measure  to  survey  a  system’s  quality.  TQM  is  a  management  philosophy 
providing  tools  and  techniques  for  managers  to  follow  for  improved  quality  systems. 
(1,4,56)  TQM  is  consistent  with  the  Single  Process  Initiative  set  forth  by  Dr.  Gansler  in 
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his  1995  memorandum  aligning  best  commercial  practices  to  be  better  reflected  in  DoD 
acquisition  and  procurement  activities  (17).  The  quality  management  process  shall  be 
capable  of  the  following  key  activities:  (17,23) 

•  Establish  capable  processes 

•  Continuously  improve  processes 

•  Monitor  and  control  critical  processes  and  product  variation 

•  Establish  mechanisms  for  field  product  performance  feedback 

•  Implement  an  effective  root-cause  analysis  and  corrective  action  system 
Achievement  of  quality  requires  an  effective  quality  management  process  in 

conjunction  with  effective  business  and  technical  practices.  Achievement  requires 
engineering  and  manufacturing  practices  that  emphasize  robust  design  along  with 
enterprise- wide  process  maturity  through  continuous  process  improvement  efforts. 
Benefits  include  first  time  pass  quality,  decreased  cycle  time,  as  well  as  reductions  in 
rework,  engineering  changes,  and  inspections.  These  benefits  translate  into  improved 
affordability  and  reduced  production  transition  risk.  (4,13,56) 

D.  THE  RELATIONSHIP  BETWEEN  DEFENSE  CONTRACTORS  AND 
DOD 

The  change  in  quality  assurance  philosophy  from  MIL-Q-9858  to  allowing 
contractors  to  use  equivalent  commercial  quality  assurance  systems  and  ISO  9000 
standards  has  program  and  contract  management  implications.  Some  of  these 
implications  are  discussed  under  the  Advantages  and  Barriers  section  of  this  chapter.  All 
those  questioned  agree  the  DoD  SPI  and  use  of  the  “most  effective  quality  system” 
continues  to  allow  a  better  quality  environment.  SPI  and  “most  effective  quality  system” 
do  not  come  without  issue. 

The  key  issue  is  best  explained  by  one  of  the  respondents  and  addresses  an 
overarching  DoD  problem  facing  industry  and  the  DoD  in  instituting  SPI  and  “most 
effective  quality  system.” 

The  SPI  of  only  one  standard  to  industry  is  deceiving.  MIL-Q  and  MIL- 1 
were  not  canceled  [sic].  They  are  not  the  preferred  method  according  to 
DoD  policy.  So  when  we  (DoD)  stopped  dictating  what  quality  system  to 
use  to  allow  the  “most  effective  quality  system,”  we  opened  the  door  for 
industry  to  choose  the  system.  Many  organizations  decided  to  remain  with 
the  MIL-Q  and  MIL-I  standards.  DCMA  now  must  not  only  understand 
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ISO  9000:2000  but  all  of  the  previous  military  standards  still  in  use,  many 
industry  accepted  quality  standards  like  QS  9000  [that]  the  automobile 
industry  uses,  as  well  as  ISO  9000:1994.  The  way  the  FAR  is  written  we 
are  discouraged  to  dictate  to  an  organization  any  of  these  systems  as  long 
as  they  meet  the  “most  effective”  criteria.  The  challenge  of  understanding 
all  of  these  systems,  although  there  are  many  similarities  between  them,  is 
difficult  for  DCMA  to  consistently  and  to  appropriately  manage  across  the 
numerous  sectors  of  business  DoD  is  contracted  with. 

DoD  suppliers’  complaint  about  the  cost  of  having  two  or  more  systems 
prior  to  SPI  is  only  a  half-truth.  Most  organizations  only  used  one  system, 
MIL-Q;  the  cost  of  quality  is  something  an  organization  will  commit  to 
even  if  it  is  not  contractually  outlined.  The  cost  of  implementing  ISO,  if 
they  chose  to  leave  MIL-Q,  is  only  a  5%  to  10%  increase.  The  differences 
between  ISO  9000-1994  and  MIL-Q  9858  were  not  significant  enough,  in 
my  opinion,  to  warrant  significantly  more  cost.  Suppliers  wanted  DoD 
inspectors  out  of  their  factories  and  in  SPI  industry  saw  a  way  to 
accomplish  the  elimination  of  in-house  DCMA  representatives.  Industry 
and  NATO’s  push  to  use  ISO  9000:1994  instead  of  the  Allied  Quality 
Assurance  Procedures  (AQAPs)  (see  Appendix  C)  pushed  DoD  to  make 
the  change  to  allow  QMS  like  ISO. 

One  proposed  solution  to  address  the  numerous  QMS  that  DoD  organizations 
such  as  DCMA  must  be  versed  in  is  to  mandate  the  use  of  ISO  9000.  All  respondents 
agreed  that  ISO  should  not  be  imposed.  The  reason  stated  predominately  was  the  burden 
it  would  place  on  small  businesses.  Small  business  may  choose  to  implement  ISO.  The 
primary  burden  is  the  financial  and  resource  cost  of  third  party  registration  (16).  DoD 
does  not  mandate  third  party  registration,  only  the  existence  of  an  adequate  QMS.  The 
intent  of  ISO  9000,  like  other  quality  systems,  is  to  add  value  to  the  organization  through 
enhanced  quality  and  process  control.  The  expectation  is  once  a  system  like  ISO  9000  is 
implemented,  organizational  effectiveness  and  efficiency  resulting  in  better  processes  and 
products  absorb  implementation  costs. 

Assume  for  discussion  that  DoD  mandates  use  of  ISO  9000,  and  more  specifically 
requires  ISO  9000  with  third  party  registration.  A  DoD  ISO  9000  registration  mandate 
could  be  very  problematic  and  could  result  in  a  breakdown  of  the  arms- length 
relationship  between  DoD,  registrars  of  ISO,  and  contractors.  The  view  is  based  on  the 
underlying  issues  raised  in  many  of  the  respondents’  comments  but  one  of  the 
interviewees  described  it  best: 
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If  DoD  chooses  to  mandate  ISO,  the  only  way  to  make  it  work  would  be  to 
select  one  registrar  to  handle  all  DoD  contracts.  This  would  be  required  to 
ensure  a  level  playing  field  among  DoD  contractors. 

There  is  also  a  problem  with  picking  only  one  registrar.  Registrars  become 
financially  tied  to  the  successes  and  failures  of  the  companies  they  evaluate.  The  success 
of  the  registrar’s  business  is  tied  to  registration,  and  registrars  only  get  return  business  if 
they  register  an  organization.  Independent  registrars  are  not,  in  fact,  independent.  To 
ensure  this  conflict  of  interest  would  not  occur,  DoD  would  need  to  pay  the  selected 
registrar  for  all  DoD  business,  instead  of  having  the  private  organization  or  contractor 
assume  the  cost  of  implementing  a  quality  system.  Currently,  DoD  can  benefit  from  a 
company’s  quality  system  without  absorbing  all  of  the  cost  for  the  implementation  of  that 
system.  The  cost  of  the  periodic  and  third  year  audits  would  be  too  high  for  DoD  to 
absorb  and  the  conflict  of  interest  issue  is  not  easily  resolved.  In  one  quality  manager’s 
opinion: 

DoD  cannot  use  the  blanket  policy  of  choosing  only  ISO  registered 
organizations  to  do  business  with.  Commercial  buyers  and  producers 
must  be  better  than  that  (ISO).  Selection  must  be  process  and  skill 
determined  and  situation  dependent.  You  want  to  use  source  inspection 
and  performance  data  and  feed  that  into  a  process  of  source  selection. 

DCMA  and  DoD  have  not  done  a  good  job  of  doing  that  they  have  not 
invented  the  proper  tools  to  make  that  process  work  as  well  as  industry 
has.  A  good  example  of  how  this  works  is  a  vendor-rating  program. 

Product  is  inspected;  performance  data  is  generated  and  processed  into  our 
vendor-rating  program  across  the  organization.  We  use  it  as  a  tool  for 
future  supplier  selection.  I  see  DCMA  request  the  data  but  I  don’t  see  a 
documented  transfer  or  process  to  provide  the  information  across  DoD 
organizations. 

Under  the  current  regulation,  the  DoD  reaps  the  rewards  of  industry  applying  their 
own  QMS  without  burdening  that  cost  on  Government  contracts.  DCMA’s  responsibility 
is  to  assure  the  system  best  suites  the  needs  of  the  customer  and  that  the  overhead  rates 
properly  reflect  industry  cost. 

The  DoD  acquisition  workforce  must  currently  be  versed  in  numerous 
commercial  and  military  based  quality  management  systems.  Education  and  training 
cannot  only  focus  on  quality  standards  such  as  ISO  9000  and  ANSI  9000  series,  but  on 
industry  and  market  specific  quality  systems  as  well,  including  military  QMS.  The 
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education  and  training  of  the  DoD  acquisition  workforce  is  more  critical  under  SPI. 
Specifically,  ISO  9000:2000  training  is  currently  lacking. 

The  DCMA  organizational  view  is  that  ISO  has  not  undergone  a  significant 
change.  Accordingly,  the  agency  is  not  sponsoring  training  on  the  change,  beyond 
conducting  an  internal  review  and  dissemination  of  the  ISO  9000:2000  Standard.  DCMA 
is  encouraging  local  commands  to  send  QMS  lead  auditors  to  training,  to  conduct  “train 
the  trainer”  instruction.  Once  lead  auditors  are  trained  they  will  train  the  local 
organizations  on  ISO  9000:2000  changes.  The  problem  associated  with  this  strategy  is 
that  DCMA  field  activities  work  under  limited  budgets.  If  DCMA  as  an  organization 
does  not  recognize  the  importance  of  funding  training  from  a  corporate  level,  then  what 
emphases  will  local  DCMA  Commanders  place  on  their  employees  receiving  adequate 
training  equivalent  to  what  their  industry  counterparts  are  receiving?  The  lack  of  DCMA 
funding  to  address  this  issue  may  result  in  inconsistent  and  inadequate  audits  of  industry 
using  ISO  9000:2000  as  their  QMS.  One  DoD  employee  surmised. 

The  DCMA  Headquarters  view  is  [ISO  9000]  2000  is  not  significantly 
different  from  [ISO  9000:]  1994.  The  civilian  sector  sees  this  [ISO 
9000:2000]  differently  and  is  committing  resources  to  make  the  change. 

DCMA  should  follow  suite,  at  least  partially,  to  assure  our  auditors  are 
able  to  perform  an  audit  at  an  appropriate  level  of  knowledge  and 
expertise. 

Defense  industry  quality  assurance  managers  interviewed  believe  that  more 
resources  should  be  placed  in  training  DCMA  representatives  on  ISO  9000:2000. 
DCMA’s  ability  to  adequately  audit  and  provide  surveillance  for  the  DoD  customer  is 
directly  correlated  to  product  quality  and  performance,  which  results  in  long  term  cost 
savings  over  the  life  of  the  product  or  service.  A  few  well- spent  dollars  for  training  may 
result  in  saving  resources  and  higher  quality  products  for  DoD  users  resulting  in 
decreased  cost  and  greater  reliability  over  a  system’s  life  cycle. 

Training  and  education  is  more  difficult  with  numerous  QMS  for  the  workforce  to 
become  familiar  with.  Previously  MIL-Q-9858  and  MIL-I-45208  provided  detailed  and 
prescriptive  guidance  in  the  form  of  two  handbooks,  commonly  referred  to  as  H50  and 
H51  (38,39).  The  handbooks  provided  cases  and  examples  for  the  auditor  of  MIL-Q- 
9858  to  use  when  conducting  audit  or  when  approached  with  varying  audit  and 

55 


surveillance  circumstances.  No  such  handbook  exists  for  DCMA  to  use  when  applying 
the  numerous  audit  functions  it  is  asked  to  perform  of  military  and  commercial  QMS. 
Handbooks  50  and  51  are  used  with  MIL-Q-9858  and  MIL- 1-45208,  respectively.  The 
need  for  case  studies  and  audit  surveillance  training  and  experience  remains.  In  the  thesis 
writer’s  research,  no  library  of  consolidated  lessons  learned  is  available  across  DoD  or 
DCMA  to  serve  and  to  share  the  daily  wealth  of  knowledge  and  experience  gained  by  the 
acquisition  workforce  in  administering  and  managing  QMS.  An  interactive  and  integrated 
approach  incorporating  best  practices  and  lessons  learned  would  aid  DoD  and  industry  in 
establish  a  consistent  approach  to  DCMA  audit  and  surveillance. 

Training  and  education  are  central  to  cultural  change.  The  introduction  of  SPI 
and  the  “most  effective  quality  system”  is  slowly  changing  the  cultural  relationship  of  the 
DoD  and  defense  industry.  DCMA  culture,  in  many  aspects,  is  no  different  than  the 
culture  in  many  other  public  or  private  organizations.  Culture  is  a  most  difficult  concept 
to  understand  and  is  often  difficult  to  change.  Without  the  ability  to  change,  organizations 
in  the  private  sector  lose  their  competitiveness;  in  the  public  sector,  organizations  that 
will  not  change  soon  lose  their  relevance. 

In  the  public  arena  measures  of  success  and  performance  are  often  ambiguous  or 
conflicting,  with  no  one  person  controlling  the  agenda  of  the  organization,  and  DCMA  is 
no  exception.  This  is  not  to  say  organizational  change  cannot  occur.  The  process  of 
cultural  change  in  public  organizations  is  different  in  that  leaders  of  the  organizations 
cannot  provide  rewards  for  cultural  change  without  budgetary  authority,  or  alternatively 
laws  and  regulations  may  dictate  how  rewards  are  given  to  public  organization  members. 
Without  the  ability  to  match  cultural  change  to  some  type  of  incentive  cr  reward  for 
changing,  a  catalyst  does  not  exist  to  make  the  organization’s  stakeholders  adopt  any 
significant  change.  Leaders  of  public  organizations  often  must  be  much  more  creative  in 
incentivizing  change,  for  without  incentive  an  organization’s  culture  will  remain  the 
same.  One  DCMA  quality  manager  spoke  about  the  DCMA  view  toward  ISO: 

In  1988  DoD  wanted  nothing  to  do  with  ISO  9000.  The  culture,  while 
slowly  changing,  still  has  yet  to  wholly  embrace  the  SPI  and  ISO  9000. 

Culture  is  difficult  to  change.  The  aging  of  our  workforce,  the  decline  in 
our  quality  assurance  workforce  numbers  from  24,000  in  1995  to  12,000 
currently  [numbers  are  approximate]  and  our  inability  to  attract  some  of 
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the  best  talent  in  the  field  of  quality  assurance  and  management  are  factors 
affecting  the  DCMA  culture. 

The  dominant  view  is  that  DCMA  culture  has  changed,  but  still  has  some  areas  to 
improve.  Of  greatest  concern  to  industry  was  the  “graying”  DCMA  workforce  trained 
under  the  detail  specifications  and  adversarial  relationships  between  DoD  and  industry  of 
the  past.  The  difficulty  of  DCMA  to  attract  entry-level  and  apprentice  employees  to  its 
workforce  is  a  Government -wide  issue.  Positions  within  the  Federal  Government  are  an 
unattractive  alternative  to  careers  in  the  private  sector,  given  the  high  employment  rates 
of  the  1990s.  Federal  workforce  reductions,  particularly  those  in  the  DoD,  eliminated 
many  of  the  junior  DoD  employees  through  Federal  protection  of  senior  DoD  workforce 
members,  causing  a  potentially  severe  problem  in  the  ability  of  the  DoD  civilian 
workforce  to  address  future  missions.  One  defense  industry  observer  said, 

. .  .DCMA  reduced  their  personnel  by  two-thirds  over  the  past  decade  [and] 
all  you  had  left  was  older,  more  senior  employees.  These  employees  come 
from  a  past  culture  of  inspection  and  monitoring,  [reinforcing  the  position] 
that  we  (DCMA)  cannot  trust  the  contractor.  You  reach  a  point  where  it  is 
very  difficult  for  DCMA  to  change  the  culture.  It  is  a  generational  issue. 

They  (DCMA)  are  being  taught,  trained,  and  told  to  do  it  differently  but 
can’t  or  won’t  change  due  to  the  past  cultural  influence.  The  environment 
has  become  increasingly  more  technical  with  change  occurring  more 
rapidly  than  in  the  past. 

According  to  some  DoD  officials  industry  pushed  for  the  issuance  of  the  Single 
Process  Initiative  (SPI)  in  hopes  to  reduce  and  or  eliminate  the  amount  of  DCMA 
oversight  and  surveillance  (23,59).  What  SPI  failed  to  eliminate  for  industry,  severe 
DCMA  budgetary  and  resource  constraints  did.  DCMA  oversight  and  surveillance  little 
resembles  what  it  was  in  the  1980s.  SPI  and  resource  constraints  have  forced  DCMA  to 
refocus  its  audit  and  surveillance  techniques  from  oversight  to  insight.  The  researcher 
does  not  see  this  as  a  weakness  of  DCMA  but  rather  a  challenge  to  continue  to  ensure 
products  conform  to  the  contract  performance  specifications  within  the  limits  of  public 
funds. 

Some  industry  respondents  argue  that  DCMA  should  reduce  oversight  with  ISO 
registered  companies.  The  best  approach  seems  to  be  based  on  various  factors,  and  third- 
party  ISO  registration  is  one.  DCMA  must  base  the  amount  of  oversight  and  audit  it 
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conducts  on  accurate  estimates  of  the  risk  associated  with  the  contract  and  DCMA’s 


quality  assurance  role  should  focus  on  this  area.  DCMA  cannot  become  detached  from 
the  customer’s  product  requirements.  The  DCMA  workforce  needs  a  clear  understanding 
of  customer  requirements  due  to  their  contract  oversight  responsibilities.  DCMA  ACOs 
and  QARs  are  the  first  and  often  only  representation  the  user  has  on  the  production  floor. 
Here  lies  significant  risk  to  product  conformity  if  DCMA  fails  to  understand  and  properly 
recognize  customer  requirements. 

Under  constrained  resources  the  DoD  should  make  use  of  its  risk  management 
system  to  evaluate  what  level  of  contract  oversight  is  appropriate.  DoD  organizations 
like  DCMA  should  consider  internal  organizational  competencies  and  strengths  as  part  of 
their  risk  management  evaluation  of  defense  contractors.  Many  defense  industry  quality 
managers  observed  the  following  excerpt: 

The  registrars  are  head- and- shoulders  above  DCMA  on  their  ability  to 
evaluate  compliance  to  the  ISO  Standard.  They  (ISO)  are  much  more 
capable  of  understanding  the  systems  evaluation  of  an  organization  and 
the  organization’s  written  procedures  to  match  the  system.  Registrars  look 
at  the  entire  business  system.  Registrars  are  not  very  good  at  looking  at  a 
product  and  measuring  its  conformity  and  compliance  to  the  procedures. 

DCMA  is  hardware  oriented  and  much  more  familiar  with  the  hardware 
requirements. 

And  a  similar  observation: 

There  are  differences  among  registrars  and  differences  among  DCMA 
auditors.  Government  representatives  come  from  different  agencies,  from 
various  areas  and  communities,  with  varying  objectives  against  the  same 
contract.  DoD  agencies  waste  time  with  the  infighting  that  occurs 
between  the  various  DoD  stakeholders.  It  constrains  DCMA  and  all  the 
other  DoD  agencies  to  adequately  evaluate  an  organization  because  DoD 
cannot  reach  consensus  on  what  they  expect  of  the  contractor  performance 
against  the  contract.  The  roles  of  contract  administration  are  not  agreed 
upon  between  DCMA  and  the  buyer. 

The  PCO  and  ACO  must  clearly  designate  and  define  the  areas  of  contract 
administration  and  procurement  functions.  The  resident  DCMA 
representatives  should  take  the  lead  in  deciding  the  jurisdiction  of  the  PCO 
and  the  ACO  in  many  of  these  matters. 

The  DoD  must  provide  clear  guidance  to  DCMA  as  to  what  and  how  they  are  to 

be  used  as  a  tool  to  mitigate  contract  risk.  DCMA  is  not  resourced  financially  or,  in 
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manpower  (adequately  trained  and  in  numbers)  to  properly  address  all  of  the  missions  it 
is  asked  to  perform.  DoD  users  of  DCMA  must  insure  the  scope  of  audits  and 
surveillance  required  is  clearly  defined.  This  helps  ensure  resources  are  not  wasted  on 
non-value  added  tasks.  Third  party  ISO  registration  is  just  one  area,  of  many,  that 
DCMA  must  evaluate  and  the  extent  of  confidence  that  DCMA  should  place  on  the 
registration. 

There  is  not  a  cookie  cutter  answer  to  the  right  quality  assurance  program; 
approaches  vary  from  industry  to  industry  and  within  industry  sectors.  ISO  9000  is  not  a 
silver  bullet  and  once  shot  or  put  into  place,  it  does  not  necessarily  mean  output  will  be 
flawless.  Oversight  and  audit  personnel  must  then  build  a  broad  base  of  knowledge, 
experience,  and  efficiency  in  performing  and  carrying  out  oversight  and  audit  functions. 
Auditors  and  quality  assurance  specialists  must  go  to  the  shop  or  factory  floor  and  speak 
to  the  employees  that  actually  put  quality  into  the  product  through  their  own  skill, 
knowledge,  and  pride  in  the  outcome  of  their  respective  manufactured  item  or  service. 

Market  and  industry  analysis  and  research  is  key  to  making  a  proper  assessment 
of  a  quality  processes  existence  in  whatever  area  the  auditor  or  contracting  officer  is 
working.  Armed  with  this  knowledge,  oversight  and  audit  functions  can  carry  out  their 
analysis  effectively  resulting  in  little  interference  creating  an  environment  where 
Government  auditor  and  contractor  team  develop  and  produce  the  best  value  outcomes 
for  both  industry  and  Government.  System  managers,  contracting  officers  as  well  as 
auditors  of  quality  management  must  make  a  quality  assurance  determination.  The  costs 
and  benefits  of  a  contractor’ s  quality  management  system  must  be  evaluated  to  determine 
that  the  quality  system  that  will  produce  the  desired  outcome  sought  by  the  buyer.  This 
determination  must  be  made  on  a  contract- by- contract  basis.  (3,9,21) 

Some  observers  advocate  that  successful  completion  of  ISO  registration  should  be 
equivalent  to  a  satisfactory  preaward  survey.  Numerous  problems  are  associated  with  this 
assertion,  which  brings  to  light  many  of  the  weaknesses  of  the  ISO  9000  certification. 
Proponents  argue  ISO  auditors  use  many  survey  procedures  that  are  almost  exactly  the 
same  as  are  used  in  preaward  surveys.  While  to  some  extent  this  is  true,  the  ISO  audit  for 
registration  occurs  once,  with  cursory  audits  every  6  months  performed  by  ISO 
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registration  teams.  Complete  reviews  are  carried  out  only  after  three  years  of  being 
registered  as  ISO  9000  compliant  quality  organizations  (9,53).  This  frequency  and  depth 
may  not  be  enough  to  ensure  a  quality  system  meeting  the  needs  of  a  particular 
procurement. 

As  was  mentioned  earlier,  every  quality  system  must  be  evaluated  on  a  case-by¬ 
case  basis  as  it  applies  to  the  given  procurement  circumstance.  For  example,  assume  that 
the  US  Army  has  contracted  with  a  company  to  manufacture  a  replacement  for  its  aging 
medium  truck  fleet.  One  of  the  contractors  in  the  competitive  range  is  an  ISO  9000 
certified  company  but  has  not  produced  a  truck  of  this  type  and  series  since  World  War 
II.  It  would  be  advisable,  and  appropriate  within  the  Standard  Inspection  Clause,  to 
review  the  company’s  current  quality  system.  Concerns  throughout  industry  mention  that 
registered  auditors  supposedly  qualified  by  the  ISO  fail  to  provide  adequate  guidance  and 
measures  in  certifying  organizations  as  ISO  9000  compliant.  (3,14,37,53). 

Who  then  will  be  able  to  distinguish  the  difference  between  good  or  poor  ISO 
9000  auditors  if  not  through  the  Government’s  own  oversight  and  auditing  groups?  FAR 
Part  46.202  prescribes  that  the  Government  “bears  the  responsibility  of  determining 
whether  the  standard  is  implemented  properly  and  adequately,”  not  the  ISO.  One 
interviewee  whose  views  were  representative  of  the  majority  opinion  on  DCMA 
oversight  offered: 

The  amount  of  surveillance  or  oversight  should  not  be  based  on  any 
redundancies  found  between  registrars  and  DCMA  but  driven  more  by 
how  well  the  process  is  working  and  measuring  up  to  the  oversight  DCMA 
feels  they  need  to  conduct  and  as  to  how  we  are  evaluated  to  contractual 
performance  measures.  The  relationship,  if  one  exists,  is  if  ISO  9000  has 
helped  us  to  improve  and  organize  our  process  to  a  point  where  DCMA  is 
more  confident,  not  the  ISO  registration.  It  is  a  matter  of  confidence  and 
risk  associated  with  the  contract,  not  a  correlation  of  registrars  and 
DCMA. 

The  Government  would  assume  significant  risk  in  delegating  audit  and  oversight 
authority  to  any  outside  organization.  The  Government  risks  losing  the  opportunity  to 
guide  and  mentor  contractors  through  the  acquisition  cycle  and  its  association  with  the 
respective  contractor.  The  Government  would  also  risk  losing  valuable  insight  required 
to  maintain  the  capability  for  monitoring  and  establishing  a  base  from  which  future 
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quality  assurance  auditing  must  occur.  ISO  9000  certification  should  be  promoted  and  is 
encouraged  by  DoD.  While  ISO  9000  is  no  guarantee  of  a  quality  process  or  product  it 
provides  a  certain  level  of  confidence  towards  meeting  requirements  in  quality  assurance 
the  Government  might  expect  inherent  to  a  contractor’s  quality  system  process. 

Publicity  concerning  low  quality  can  ruin  the  reputation  of  a  company  and  can 
have  the  same  effect  on  any  DoD  program  where  public  trust  is  involved  and  expected  in 
any  procurement  (3,51).  DoD  has  worked  very  hard  over  the  past  decade  to  reverse  a 
loss  of  public  trust  associated  with  some  procurement  activities  in  the  1980’s  (23,44). 
DoD  must  be  aware  that  suppliers  that  are  QS  9000  certified,  such  as  Firestone,  may  pass 
the  surveillance  audit,  but  the  customer  is  still  not  guaranteed  protection  from  defects  (9). 
In  the  public  sector,  the  Government  is  held  accountable  for  the  actions  of  its  suppliers. 
Only  when  the  contractor  and  Government  take  proactive  efforts  can  a  truly  effective  and 
efficient  quality  management  system  work. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

ISO  9000:  2000  assimilates  the  20  elements  of  ISO  9001:1994.  The  eight  quality 
management  principles  and  five  disciplines  of  ISO  9000:2000  incorporate  the  essence  of 
the  20  elements.  ISO  intent  is  for  the  2000  version  to  address  shortcomings  of  the  1994 
version  and  to  update  ISO  to  current  quality  management  and  assurance  practices  and 
techniques. 

A  lack  of  understanding  of  the  limitations  and  capabilities  of  ISO  9000  exists 
within  the  DoD  acquisition  workforce  and  defense  contractors.  Neither  ISO  9000:1994 
nor  ISO  9000:2000  guarantees  high  quality  product  if  the  QMS  system  is  not  tailored  to 
the  product  or  the  workforce  fails  to  follow  the  processes  and  practices  of  the  QMS.  The 
ISO  9000:2000  focus  on  management  involvement,  process  quality  management, 
continual  improvement  and  customer  satisfaction  recognizes  the  essentiality  of 
management  involvement.  ISO  9000:2000  provides  guidelines  for  building  an 
environment  that  encourages  and  supports  the  process  outcomes  desired  by  the  customer. 
Whereas  ISO  9000:2000  is  no  silver  bullet,  it  will  help  to  achieve  a  clear  understanding 
of  customer  requirements,  emphasis  on  quality  from  the  front-line  workforce  to  corporate 
executives,  and  proven  procedures  monitored  by  appropriate  metrics  which  allow  for 
contractual  improvement  arm  an  organization  with  the  tools  to  consistently  produce 
conforming  products  and  thus  higher  customer  satisfaction. 

ISO  9000:2000  can  provide  the  foundation  and  framework  for  a  quality- achieving 
organization  but  without  the  materials  (the  policies,  the  techniques,  the  procedures,  and 
the  metrics)  discussed  in  Chapter’s  II- IV,  all  that  remains  is  a  frame  of  a  house,  an  empty 
shell,  never  realizing  its  QMS  potential. 

The  role  of  DCMA,  and  other  DoD  contract  management  activities,  is  critical  in 
an  environment  of  misplaced  trust  and  misunderstood  capabilities.  DCMA  is  the  DoD 
ombudsman,  the  frontline  worker,  and  the  advocate  who  provides  QA  oversight  and 
inspection  of  products  destined  for  the  Department  of  Defense.  Quality  assurance 
specialists  assigned  to  DCMA  have  the  key  role  as  the  Government’s  quality  assurance 
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representative  who  approves  the  quality  management  system  for  use  in  producing  a 
particular  warfighting  system.  Application  of  ISO  9000:2000  must  meet  their  standards. 

B.  CONCLUSIONS 

The  list  that  follows  is  a  consolidation  of  the  conclusions  resulting  from  this  thesis 
research. 

•  DCMA  role  is  more  complex  under  the  SPI.  Under  the  SPI  DCMA  is 
responsible  to  evaluate  numerous  QMS.  Prior  to  1996,  with  few 
exceptions,  it  was  only  the  MIL-Q-  Standards. 

•  ISO  provides  a  baseline  or  framework  for  a  QMS.  Culture,  manufacturing 
ability,  technological  know-how,  leadership,  management  emphasis,  and 
process  quality  control  and  management  metrics  provide  the  highest  level 
of  confidence  and  reduction  of  risk  associated  with  a  procurement. 
However,  ISO  registration  does  not  guarantee  to  the  DoD  that  defense 
contractors  will  manufacture  a  conforming  product  or  even  understand  the 
customer’s  requirement  through  the  contract  and  SOW. 

•  ISO  9000:2000  contains  the  necessary  elements  of  a  quality  management 
system  that,  if  correctly  applied,  is  suitable  for  higher- level  quality 
requirements.  In  every  case  the  QMS  must  be  examined  against  the 
quality  requirements  of  the  product  itself.  This  is  accomplished  in 
accordance  with  the  provisions  of  the  Standard  Inspection  Clauses  used  in 
all  Government  contracts. 

•  ISO  9001:2000  principles  and  disciplines  integrate  the  20  elements  of  ISO 
9001:1994.  The  integration  reduces  a  stovepipe  approach  to  QMS 
implementation  and  utilization.  The  ISO  9000:2000  takes  a  more 
proactive  approach  than  ISO  9000:1994,  considering  as  organization’s 
culture  and  environment,  leadership  attention,  and  customer. 

C.  RECOMMENDATIONS 

The  list  that  follows  is  a  consolidation  of  the  recommendations  based  on  the 
conclusions  and  information  gathered  from  this  thesis  research. 

•  The  Department  of  Defense  should  use  ISO  9001:2000  as  its  benchmark 
Quality  Management  System  for  use  when  a  higher- level  quality  is 
required. 

•  The  Defense  Acquisition  University  and  applicable  service  schools  should 
teach  ISO  9000:2000  as  the  quality  management  system  baseline. 
Additionally,  these  schools  should  describe  the  application  of  ISO 
9000:2000  in  DoD  contracts. 

•  Buying  activities  should  work  closely  with  DCMA  to  ensure  a  systematic 
approach  for  contractual  implementation  of  ISO  9001:2000 


64 


DCMA  should  continue  its  current  practice  of  issuing  a  Statement  of 
Qualification  to  contractors  found  in  compliance  with  higher- level 
contract  quality  system  standards,  e.g.,  ISO  9001:2000. 

Interpretation  of  ISO  9001:2000  language  should  also  be  addressed 
through  internal  DCMA  guidance  and  DAU  training. 

DCMA  in  conjunction  with  DAU  schools  should  establish  a  virtual  library 
for  use  Do  D- wide,  capable  of  providing  the  instructional  tools  that 
Handbooks  50  and  51  gave  the  acquisition  workforce  under  MIL-Q-9858 
and  MIL-Q-45208. 

DCMA  should  maintain  a  library,  which  contains  a  compilation  of  defense 
contractors  who  have  used  ISO  9000:2000,  their  Registrars,  contractors’ 
history  of  QMS  performance,  and  lessons  learned.  This  resource  would 
provide  current  and  future  acquisition  employees  information  upon  which 
to  base  future  ISO  9000  QMS  decisions. 

USD  (AT&L)  should  establish  a  policy  that  addresses  current  and  future 
QMS  updates  specifically  focusing  on  contractual  aspects  of  QMS 
updates. 

DCMA  should  continue  to  update  its  QMS  checklist  to  reflect  changes 
implemented  by  ISO  9000:2000. 
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APPENDIX  A.  LIST  OF  COMPANIES  PARTICIPATING  IN 

INTERVIEWS 


AM  General 
Boeing 

General  Dynamics 
Honeywell 
Lockheed  Martin 
Motorola 

Northrop-  Grumman 

Oshkosh 

Raytheon 

Stewart  and  Stevenson 
TRW 

United  Defense 

DCMA-East 

DCMA-HQ 
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APPENDIX  B.  ISO  TO  MIL-Q  CROSS  REFERENCE 


MIL-Q- 985  8 

94-9001 

94-9002 

94-9003 

94-9004 

1.1,  1.2 

0 

0 

0 

1 

1.3 

4.2 

4.2 

4.2 

5 

2 

0 

0 

0 

0 

3.1 

4. 1.2.1 

4. 1.2.1 

4. 1.2.1 

4/5. 5/5. 2.3 

3.2 

4.3 

4.3 

0 

5.3.3/8.3/8.25 

3.3 

4.9.1 

4.8.1 

0 

11.3/17.2 

3.4 

4.16 

4.15 

4.4/4. 6 

5.3.4/17.3 

3.5 

4.14 

4.13 

0 

15 

3.6 

0 

0 

0 

43.2/6 

4.1 

4.5 

4.4 

4.3 

17.2 

4.2 

4.11 

4.10 

4.6 

13 

4.3 

4.11 

4.10 

0 

0 

4.4 

4.11 

4.10 

0 

0 

4.5 

4.2 

4.2 

0 

0 

5.1 

4.6 

4.5 

0 

9 

5.2 

4.6.3 

4.5.3 

0 

9.5 

6.1 

4.10 

4.9 

0 

11.2/12.1 

6.2 

4.9 

4.8 

0 

10/12.2 

6.3 

4.10.3 

4.9.3 

4.5 

12.3 

6.4 

4.15 

4.14 

4.9 

16 

6.5 

4.13 

4.12 

4.8 

14 

MIL-Q-9858 

94-9001 

94-9002 

94-9003 

94-9004 

6.6 

4.2 

4.18 

4.12 

20 

6.7 

4.12 

4.11 

4.7 

11.7 

7.1 

0 

0 

0 

0 

7.2 

4.7 

4.6 

0 

0 

8 

0 

0 

0 

1 

Table  3.  MIL-Q-9858  to  ISO  9000: 1994  Family,  [After  Ref.  3], 
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ISO/FDIS  9001:2000 

ISO  9001:1994 

1  Scope 

1 

1.1  General 

1.2  Application 

2  Normative  references 

2 

3  Terms  and  definitions 

3 

4  Quality  management  system  [title  only] 

4.1  General  requirements 

4.2.1 

4.2  Documentation  requirements  [title  only] 

4.2.1  General 

4.2.1  +  4.5.1 

4.2.2  Quality  manual 

4.2.1 

4.2.3  Control  of  documents 

4.5.1  +4.5.2 +  4.5.3 

4.2.4  Control  of  quality  records 

4.16 

5  Management  responsibility  [title  only] 

5.1  Management  commitment 

4.1  +  4. 1.2. 2  +  4.2.1 

5.2  Customer  focus 

4.3.2 

5.3  Quality  policy 

4.1.1 

5.4  Planning  [title  only] 

5.4.1  Quality  objectives 

4.1.1  +  4.2.1 

5.4.2  Quality  management  system  planning 

4.2.3 

5.5  Responsibility,  authority  and  communication  [title  only] 

5.5.1  Responsibility  and  authority 

4. 1.2.1 

5.5.2  Management  representative 

4. 1.2.3 

5.5.3  Internal  communication 

5.6  Management  review  [title  only] 

4.1.3 

5.6.1  General 

4.1.3 
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ISO/FDIS  9001:2000 

ISO  9001:1994 

5.6.2  Review  input 

4.1.3 

5.6.3  Review  output 

4.1.3 

6  Resource  management  [title  only] 

6. 1  Provision  of  resources 

4. 1.2.2 

6.2  Human  resources  [title  only] 

6.2.1  General 

4.1.2.2  +  4.2.3  +  4.18 

6.2.2  Competence,  awareness  and  training 

4.18 

6.3  Infrastructure 

4. 1.2. 2  +  4.9 

6.4  Work  environment 

4.9 

7  Product  realization  [title  only] 

7.1  Planning  of  product  realization 

4.2.3  +  4.9  +  4.10.1 

7.2  Customer-related  processes  [title  only] 

7.2.1  Determination  of  requirements  related  to  the  product 

4.3.2  +  4.4.4 

7.2.2  Review  of  requirements  related  to  the  product 

4.3.2 +  4.3.3  +4.3.4 

7.2.3  Customer  communication 

4.3.2 

7.3  Design  and  development  [title  only] 

7.3.1  Design  and  development  planning 

4.4.2 +  4.4.3  +4.4.6 
+  4.4.7  +  4.4.8 

7.3.2  Design  and  development  inputs 

4.4.4 

7.3.3  Design  and  development  outputs 

4.4.5 

7.3.4  Design  and  development  review 

4.4.6 

7.3.5  Design  and  development  verification 

4.4.7 

7.3.6  Design  and  development  validation 

4.4.8 

7.3.7  Control  of  design  and  development  changes 

4.4.9 

ISO/FDIS  9001:2000 

ISO  9001:1994 

7.4  Purchasing  [title  only] 
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7.4.1  Purchasing  process 

4.6.2 

7.4.2  Purchasing  information 

4.6.3 

7.4.3  Verification  of  purchased  product 

4.6.4  +  4.10.2  + 
4.10.3+4.10.4 

7.5  Production  and  service  provision  [title  only] 

7.5.1  Control  of  production  and  service  provision 

4.9  +  4.10.3  +  4.15.6 
+  4.19 

7.5.2  Validation  of  processes  for  production  and  service 
provision 

4.9 

7.5.3  Identification  and  traceability 

4.8  +  4.10.5  +  4.12 

7.5.4  Customer  property 

4.7 

7.5.5  Preservation  of  product 

4.15.2  +  4.15.3  + 

4.15.4  +.4.15.5  + 

4.15.6 

7.6  Control  of  monitoring  and  measuring  devices 

4.11.1+4.11.2 

8  Measurement,  analysis  and  improvement  [title  only] 

8.1  General 

4.10  +  4.17  +  4.20.1 

8.2  Monitoring  and  measurement  [title  only] 

8.2.1  Customer  satisfaction 

8.2.2  Internal  audit 

4.17 

8.2.3  Monitoring  and  measurement  of  processes 

4.9  +  4.17  +  4.20.1 

8.2.4  Monitoring  and  measurement  of  product 

4.10.2  +  4.10.3  + 

4.10.4  +  4.10.5  + 

4.20.1 

8.3  Control  of  nonconforming  product 

4.13.1+4.13.2 

8.4  Analysis  of  data 

4.14.2  +  4.14.3  + 

4.20 

8.5  Imp  rovement  [title  only] 

8.5.1  Continual  improvement 

4.1.3 

8.5.2  Corrective  action 

4.14.1  +4.14.2 
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8.5.3  Preventive  action 


4.14.1  +4.14. 


3 


Table  4.  ISO  9001 :2000  to  ISO  9001 : 1994,  [From  Ref.  55], 
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APPENDIX  C.  NATO  AQAP 


NATO  Quality  System  Standards  (AQAP’s) 

NATO  has  established  a  set  of  standardized  procedures  in  an  effort  to  support 
quality  acquisition  activities  between  North  Atlantic  Treaty  Organization  (NATO) 
member  nations.  This  approach  ensures  each  NATO  nation  performs  consistent  quality 
assurance  activities.  The  two  NATO  Standardization  Agreements  (STANAGs)  that 
govern  NATO  quality  assurance  activity  are  the  STANAG  4107  and  STANAG  4108. 

The  STANAG  4107,  titled  Mutual  Acceptance  of  Government  Quality  Assurance, 
sets  forth  the  terms  and  conditions  under  which  mutual  government  quality  assurance  of 
military  material  and  services  is  to  be  performed  by  the  national  authority  of  one  NATO 
country  on  request  of  another  NATO  country  or  NATO  organization.  The  STANAG 
4108,  titled  Allied  Quality  Assurance  Publications  (AQAP’s),  concerns  the  standardized 
development,  updating  and  application  of  the  AQAPs  in  the  procurement  of  NATO 
defense  material. 

The  original  AQAP’s  established  for  quality  assurances  were  numbered  1  through 
15.  These  AQAP’s  were  similar  to  documents  used  by  the  U.S.  DoD  for  quality 
assurance  activity.  AQAP-1  was  similar  to  MIL-Q-9858A,  AQAP-2  was  similar  to  MIL- 
HDBK-50,  AQAP-4  was  similar  to  MIL-I-45208A,  etc. 

In  the  early  1990’s,  NATO  recognized  that  quality  was  becoming  a  major  factor 
in  international  trade  and  that  the  ISO  standards  offered  harmonization  of  NATO  quality 
concepts  and  terminology  with  the  European  Community  and  other  countries  world- wide. 
NATO  decided  to  revise  the  existing  AQAP  standards  in  order  to  embrace  these  new 
international  quality  standards. 

In  February  1993,  NATO  canceled  the  AQAP  1-15  documents  and  adopted  a  new 
series  of  AQAP’s  that  incorporate  the  ISO  9000  series  quality  system  standards. 

SUPERSEDED  OR  CANCELED  AQAP’s 


AQAP  NUMBER 

DISPOSITION 

AQAP-1 

Superseded  by  AQAP- 1 10 

AQAP-2 

Superseded  by  AQAP- 119 

AQAP- 3 

Canceled 

AQAP-4 

Superseded  by  AQAP- 130 

AQAP- 5 

Superseded  by  AQAP- 119 

AQAP- 6 

Superseded  by  ISO  10012-1 

AQAP -7 

Superseded  by  ISO  10012-1 

AQAP- 8 

Transferred  to  AC/301 

AQAP- 9 

Superseded  by  AQAP- 131 
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AQAP- 1 1  Transferred  to  AC/30 1 

AQAP- 13  Superseded  by  AQAP- 150 

AQAP- 14  Superseded  by  AQAP- 159 

AQAP- 15  Superseded  by  ISO  8402 

This  new  series  of  standards  is  sometimes  referred  to  as  the  “Century  Series” 
because  of  its  numbering  system. 

AQAP- 100  -  General  Guidance  on  NATO  QA 

AQAP- 110  -  NATO  QA  Requirements  for  Design,  Development,  and  Production 
AQAP- 1 19  -  NATO  Guide  to  AQAPs  -  1 10,  - 120  and  - 130 
AQAP- 120  -  NATO  QA  Requirements  for  Production 
AQAP- 131  -  NATO  QA  Requirements  for  Final  Inspection 
AQAP- 150  -  NATO  Quality  Assurance  Requirements  for  Software  Development 
AQAP- 159  -  NATO  Guidance  for  the  Use  of  AQAP- 150 
AQAP- 160  -  NATO  Integrated  quality  requirements  for  software  throughout  the 
life  cycle. 

AQAP- 169  -  NATO  Guidance  for  the  Use  of  AQAP- 160 
AQAP- 170  -  NATO  Guide  for  the  Delegation  of  Government  Quality  Assurance 
STANAG  4107  -  Mutual  Acceptance  of  Government  Quality  Assurance  and 
Usage  of  the  Allied  Quality  Assurance  Publications 
AC/250  -HNBK  -  The  Group  of  National  Directors  for  Quality  Assurance 

Identified  below  are  the  NATO  Contractual  AQAP’s  that  directly  reference  the 
ISO  9000  contractual  standards: 

AQAP  110:  NATO  Quality  Assurance  Requirements  for  Design,  Development, 
and  Production  directly  references  the  elements  contained  in  the  ISO  9001  standard. 

AQAP  120:  NATO  Quality  Assurance  Requirements  for  Production  directly 
references  the  elements  contained  in  the  ISO  9002  standard. 

AQAP  130:  NATO  Quality  Assurance  Requirements  for  Inspection  directly 
references  the  elements  contained  in  the  ISO  9003  standard. 

It  should  be  noted  that  there  are  differences  between  the  AQAP’s  and  ISO  9000 
quality  system  standards.  The  AQAP  standards  contain  additional  requirements 
(supplements)  to  certain  ISO  elements.  These  supplements  provide  additional  direction 
and  guidance  on  NATO  quality  system  needs. 

Governments  and  organizations  sometimes  place  contracts  with  contractors 
located  outside  their  own  country.  Contract  administrative  oversight  and  support  for  those 
contracts  may  be  available  via  established  International  Agreements  and/or 
Memorandums  of  Understanding  (MOUs)  with  the  various  nations  and  organizations. 
Contract  administrative  support  may  also  be  available  through  the  Defense  Contract 
Management  Agency  International  (DCMAI)  which  has  support  and  oversight 
capabilities  in  many  countries 
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APPENDIX  D.  ISO  FAMILY 


The  ISO  9000  Family 


The  standards,  guidelines  and  technical  reports  which  make  up  the  ISO  9000  family  and 
which  are  listed  below  are  available  separately,  or  as  collections.  The  ISO  9000 
Compendium  presents  the  ISO  9000  family  in  hard  copy  form. 


Standards  and  guidelines 

ISO  9000:2000,  Quality  management 
systems  -  Fundamentals  and 
vocabulary 


ISO  9001:2000,  Quality  management 
systems  -  Requirements 


ISO  9004:2000,  Quality  management 
systems  -  Guidelines  for  performance 
improvements 


ISO  19011,  Guidelines  on  Quality 
and/or  Environmental  Management 
Systems  Auditing  (currently  under 
development) 

ISO  10005:1995,  Quality 

management  -  Guidelines  for  quality 
plans 

ISO  10006:1997,  Quality 

management  -  Guidelines  to  quality 
in  project  management 

ISO  10007:1995,  Quality 

management  -  Guidelines  for 
configuration  management 

ISO/DIS  10012,  Quality  assurance 
reauirements  for  measurins 


Purpose 

Establishes  a  starting  point  for  understanding  the 
standards  and  defines  the  fundamental  terms  and 
definitions  used  in  the  ISO  9000  family  which 
you  need  to  avoid  misunderstandings  in  their  use. 

This  is  the  requirement  standard  you  use  to  assess 
your  ability  to  meet  customer  and  applicable 
regulatory  requirements  and  thereby  address 
customer  satisfaction. 

It  is  now  the  only  standard  in  the  ISO  9000  family 
against  which  third-party  certification  can  be 
carried. 

This  guideline  standard  provides  guidance  for 
continual  improvement  of  your  quality 
management  system  to  benefit  all  parties  through 
sustained  customer  satisfaction. 

Provides  you  with  guidelines  for  verifying  the 
system's  ability  to  achieve  defined  quality 
objectives.  You  can  use  this  standard  internally  or 
for  auditing  your  suppliers. 

Provides  guidelines  to  assist  in  the  preparation, 
review,  acceptance  and  revision  of  quality  plans. 


Guidelines  to  help  you  ensure  the  quality  of  both 
the  project  processes  and  the  project  products. 


Gives  you  guidelines  to  ensure  that  a  complex 
product  continues  to  function  when  components 
are  changed  individually. 

Give  you  guidelines  on  the  main  features  of  a 
calibration  svstem  to  ensure  that  measurements 
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Standards  and  guidelines 

equipment  -  Part  1:  Metrological 
confirmation  system  for  measuring 
equipment 

ISO  10012-2:1997,  Quality 

assurance  for  measuring  equipment  - 
Part  2:  Guidelines  for  control  oj 
measurement  of  processes 

ISO  10013:1995,  Guidelines  for 
developing  quality  manuals 

ISO/TR  10014:1998,  Guidelines  for 
managing  the  economics  of  quality 

ISO  10015:1999,  Quality 

management  -  Guidelines  for  training 


ISO/TS  16949:1999,  Quality  systems 
-  Automotive  suppliers  -  Particular 
requirements  for  the  application  oj 
ISO  9001:1994 


Purpose 

are  made  with  the  intended  accuracy. 


Provides  supplementary  guidance  on  the 
application  of  statistical  process  control  when  this 
is  appropriate  for  achieving  the  objectives  of  Part 
1. 

Provides  guidelines  for  the  development,  and 
maintenance  of  quality  manuals,  tailored  to  your 
specific  needs. 

Provides  guidance  on  how  to  achieve  economic 
benefits  from  the  application  of  quality 
management. 

Provides  guidance  on  the  development, 
implementation,  maintenance  and  improvement 
of  strategies  and  systems  for  training  that  affects 
the  quality  of  products. 

Sector  specific  guidance  to  the  application  of  ISO 
9001  in  the  automotive  industry. 


Table  5.  ISO  9000:2000  Family,  [From  Ref.  25]. 

The  US  Equivalents  to  the  ISO  Family 

ISO  9000  Is  The  Same  As  ANSI/ISO/ASQ  Q9000 
IS 09000  =  ANSFISO/ASQ  Q9000 
IS 09001  =  ANSFISO/ASQ  Q9001 
IS 09004  =  ANSFISO/ASQ  Q9004 
IS09000  is  the  international  reference 
ANSFISO/ASQ  9000  is  the  U.S.  reference  (39) 


78 


APPENDIX  E.  EXECUTIVE  AND  DOD  POLICY  GUIDANCE 


Office  of  Management  and  Budget  Circular  A- 76 

o  Federal  policy  implementing  Federal  Activities  Inventory  Reform 
Act 

o  Authorizing  the  use  of  nongovernmental  standards  and 
specifications  in  contracts  and  the  procurement  of  “off  the  shelf’ 
(commercial)  items  for  DoD  use 

o  Applicable  to  the  majority  of  executive  agencies  Government 
should  not  compete  with  its  citizens  Government  shouldn’t  carry 
on  any  activity  if  product  or  service  can  be  procured  more 
economically  from  commercial  sources 


•  USOD  Deutch  Policy  Letter  Dated  14  Feb  94 

o  Program  offices  are  authorized  to  use  ISO  ANSI/AS QC  Q9000 
Quality  Series  standards  as  follows: 

•  In  contracts  for  new  programs 

•  In  follow-on  efforts 

•  In  current  contracts  on  a  case-by-case  basis 

•  Third  party  registration  NOT  required 

•  Third  party  registration  NOT  a  substitute  for  Government 
oversight 

•  Commercial  level  quality  systems  will  be  recognized  if  able 
to  satisfy  Government’s  needs 

•  USD  Perry  Policy  Letter  Dated  29  June  94 

o  Redefines  order  of  precedence  for  specs  and  standards, 

performance  specs,  commercial  Specs 

o  MILSPECS  and  MILSTDS  (only  with  waiver) 

o  Change  in  DFAR 

o  All  MILSPECS  and  MILSTDS  related  to  management  and 

manufacturing  will  be  phased  out  suppliers  encouraged  to  use 
commercial  standards 

o  Reduce  government  oversight 

•  SECDEF  Memorandum  May  10,  1995 

o  Subject:  Use  of  Integrated  Product  and  Process  Development  and 
Integrated  Product  Teams  in  DoD  Acquisition. 
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•  Quality  products  are  best  achieved  through  integrated 
developments  of  the  product  and  its  associated 
manufacturing  and  support  processes,  through  a  systems 
engineering  approach. 

•  SECDEF  Memorandum  Dec.  6,  1995 

o  Subject:  Common  Systems  /  ISO- 9000  /  Expedited  Block 

•  DoD  policy  to  allow  use  of  single  processes  in  a 
contractor’s  facility.  Major  DoD  initiative  to  allow  industry 
to  be  more  efficient,  improve  quality  and  reduce  overall 
cost  of  acquiring  products 

•  USD  (A&T)  Memorandum  Sept.  18,  1997 

o  Subject:  “Requiring  Processes  on  Contract” 

•  DoD  should  not  require  processes  in  solicitations  and 
contracts 

•  DoD  should  rely  on  performance  based  requirements 
whenever  practicable 

•  Principal  Deputy  Under  Secretary  of  Defense  (PDUSOD)  Policy  Mr.  Dave 

Oliver  letter  March  19,  2001 

o  Subject:  ISO  9001:2000  Quality  Management  Systems,  Transition 

•  Monitor  contractor  transition  to  new  standard 

DCMA  Policy  from  DCMA  “One  Book  Supplier  Quality  Assurance  Chapter  4.4 

•  MG  Drews  letter  dated  Nov  18,  1996. 

o  DCMA  must  assure  organizations  quality  system  complies  with 

contract  requirements  and  use  existing  data  for  evaluating 

compliance  of  organizations  quality  system 

o  DCMA  must  audit  when: 

•  existing  data  unavailable  or  inadequate 

•  directed  by  customer 

•  organizations  performance  indicates 

•  system  not  in  compliance  with  contract 

•  requirements 

•  quality  system  has  been  substantially 

•  changed 

o  Evaluation  of  Quality  System  two  basic  methods: 

•  Evaluation  by  assigned  specialists 
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•  Formal  system  audit 

o  Quality  System  Evaluation  will  be  performed  on  the  quality  system 
cited  in  the  contract 

o  DCMA  Personnel  Qualifications: 

•  DAWIA  Level  II 

•  Manufacturing  and  QA  career  field,  DCMA  auditors  meet 
ISO  10011-2  requirements 

o  Quality  System  Audit 

•  Tailored  to  reflect  scope  of  contract  requirements 

•  Records  to  indicate  how  confidence  established  in  DCMA 
Audit  Checklist  used 

o  Results  of  Quality  system  audit: 

•  Notification  to  contractor 

•  Notification  to  our  customer 

o  If  Quality  System  is  found  to  be  in  compliance,  DCMA  will  issue  a 
“Statement  of  Qualification”  to  contractor 

o  DCMA  STATEMENT  OF  QUALIFICATION: 

•  Quality  System  Qualification 

•  Name  of  Company  Qualified 

•  Applicable  Standard 

•  Document  dated  and  signed  by 

•  DCMA  CMO  Commander 

•  Copy  of  document  sent  to  customer(s) 
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APPENDIX  F.  ISO  PROCESS  MANAGEMENT  APPROACH 


The  quality  management  system  described  in  the  revised  standard  is  based  on 
quality  management  principles  that  include  the  "process  approach"  and  "customer  focus.” 
ISO  9001:2000  and  ISO  9004:2000,  both,  apply  a  process  approach.  The  "20  element" 
structure  of  ISO  9001:1994  has  been  replaced  by  this  process-based  quality  management 
system  (PQM),  which  is  shown  schematically  in  Figure  3,  below. 


Figure  3.  Model  of  a  Process  Based  Quality  Management  System,  [From  Ref.48]. 

The  adoption  of  these  principles  should  provide  customers  with  a  higher  level  of 
confidence  that  products  will  meet  their  needs  and  increases  their  satisfaction  (28,48). 
ISO  9000:2000  clause  3.4.1  defines  a  "Process"  as  any  activity  or  operation,  which 
receives  inputs  and  converts  them  to  outputs  (26).  The  output  of  one  process  may  directly 
form  the  input  to  the  next  process  and  the  final  product  is  often  the  result  of  a  network  or 
system  of  processes.  Almost  all  activities  and  operations  involved  in  making  a  product  or 
providing  a  service  are  processes.  Inputs  and  outputs  may  be  tangible  or  intangible. 
Examples  of  inputs  and  outputs  may  include  equipment,  materials,  components,  energy, 

information  and  financial  resources,  among  ethers  (25).  For  organizations  to  function, 
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they  have  to  define  and  manage  numerous  inter- linked  processes.  The  systematic 
identification  and  management  of  the  various  processes  employed  within  an  organization, 
and  particularly  the  interactions  between  such  processes,  may  be  referred  to  as  the 
‘process  approach’  to  management.  The  revised  quality  management  system  standards 
promote  the  adoption  of  a  process  approach  when  developing,  implementing  and 
improving  a  QMS.  ISO  Standards  are  based  on  just  such  a  process  approach,  in  line  with 
the  guiding  quality  management  (25).  ISO  allows  the  nature  of  the  organization’s  specific 
demands  to  determine  how  to  apply  the  standards,  but  to  perform  activities  within  the 
process  appropriate  resources  have  to  be  allocated.  A  measurement  system  can  be  used  to 
gather  information  and  data  to  analyze  process  performance  and  input  and  output 
characteristics  (52). 


PROCEDURE* 

(“Specified  way  to  carryout  an  activity  or  a 
process”  -  maybe  documented  or  not) 


Input 


(hcludes 

Resources) 


1 


PROCESS 

Output 

C'Set  of  interrelated 
or  interacting  actrvities") 

EFFECTIVENESS 
OF  PROCESS  = 
#>ilityto  achieve 
desired  results 
(Focus  of  ISO 
9001:2000). 


PRODUCT 

C Result  of  a 
process") 


MONITORING  AND 
MEASUREMENT  OPPORTUNITIES 

(Before,  during  and  after  the  process) 


4  Note  -This  isthe  definition  of  'procedure”  given  in  ISO  9000:2000. 
This  does  not  necessarily  mean  one  of  the  6 
"documented  procedures"  required  by  ISO  9001 :2000 


Figure  4.  Schematic  Representation  of  a  Process,  [From  Ref.  48]. 

ISO  9001:2000  stresses  the  importance  for  an  organization  to  identify,  implement, 
manage  and  continually  improve  the  effectiveness  of  the  processes  that  are  necessary  for 
the  quality  management  system,  and  to  manage  the  interactions  of  these  processes  in 
order  to  achieve  the  organization's  objectives.  ISO  9004:2000  focuses  on  performance 
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improvements  and  guides  the  organization  beyond  the  requirements  of  ISO  9001:2000. 
ISO  9004:2000  recommends  an  evaluation  of  the  efficiency,  as  well  as  the  effectiveness 
of  the  processes. 

The  Plan- Do -Check- Act  (PDCA)  cycle  can  be  used  to  manage  those  processes. 
The  "Plan-Do-Check-Act"  cycle  was  first  developed  in  the  1920's  by  Walter  Shewhart, 
and  was  popularized  later  by  W.  Edwards  Deming.  For  that  reason  it  is  often  referred  to 
as  "The  Deming  Cycle.”  Extensive  literature  exists  about  the  PDCA  cycle  thesis  readers 
are  encouraged  to  consult  other  literature  for  a  deeper  understanding  of  the  concept 
beyond  the  following  paragraphs  the  writer  includes  (28,31). 


"Plan"  establish  the  objectives  and  processes  necessary  to  deliver 
results  in  accordance  with  customer  requirements  and  the 
organization's  policies; 

"Do"  implement  the  processes; 

"Check"  monitor  and  measure  processes  and  product  against  policies, 
objectives  and  requirements  for  the  product  and  report  the 
results; 

"Act"  take  actions  to  continually  improve  process  performance; " 

Table  6.  Table  6.  P-D-C-A  Defined,  [From  Ref.  25]. 

Within  the  context  of  a  quality  management  system,  the  PDCA  is  a  dynamic  cycle 
that  can  be  deployed  within  each  of  the  organization's  processes,  and  to  the  system  of 
processes  as  a  whole.  It  is  intimately  associated  with  the  planning,  implementation, 
control  and  continual  improvement  of  both  product  realization  and  other  quality 
management  system  processes.  Maintaining  and  continually  improving  the  process 
capability  can  be  achieved  by  applying  the  PDCA  concept  at  all  levels  within  the 
organization.  This  applies  equally  to  high-level  strategic  processes,  such  as  quality 
management  system  planning,  or  management  review,  and  to  simple  operational 
activities  carried  out  as  a  part  of  product  realization  processes.  The  Note  in  Clause  0.2  of 
ISO  9001:2000  explains  that  the  PDCA  cycle  applies  to  the  QMS  processes  as  follows 
(26,27): 

•  Plan- establish  the  objectives  and  processes  necessary  to  deliver  results  in 
accordance  with  customer  requirements  and  the  organization's  policies 
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Do -implement  the  processes 

Check- monitor  and  measure  processes  and  product  against  policies, 
objectives  and  requirements  for  the  product  and  report  the  results 

Act-take  actions  to  continually  improve  process  performance 
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•  Itmossos  to  ilaiioiislnlt: 
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■  Design  mi  levdcpmisi 

*  fuidiuunj' 
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apply  to  dl  Ic-stnl  (-I 
processes  in  fig  x 


Figure  5.  The  "PDCA"  Approach,  [From  Ref.  28]. 

A  second  important  quality  management  principle  that  is  intimately  linked  with 
the  Process  Approach  is  the  System  Approach  to  Management,  which  states  that 
"Identifying,  understanding  and  managing  interrelated  processes  as  a  system  contributes 
to  the  organization's  effectiveness  and  efficiency  in  achieving  its  objectives."  Within  this 
context,  the  quality  management  system  comprises  a  number  of  interrelated  processes. 
The  processes  needed  for  the  quality  management  system  include  not  only  the  product 
realization  processes  (those  that  directly  contribute  to  making  the  product  or  delivering 
the  service),  but  also  numerous  management,  monitoring  and  measurement  processes, 
such  as  resource  management,  communication,  internal  auditing,  management  review, 
and  other  processes  (15).  This  can  be  seen  schematically  in  Figure  3,  which  provides 
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greater  detail  of  the  kind  of  processes  that  typically  comprise  the  quality  management 
system,  divided  among  clauses  4  -  8  of  ISO  9001:2000  and  ISO  9004:2000. 


5.  T  op  M  augment  Processes 

Exampks: 

•Process  to  define  quality  policy 
and  objectives 
•  Comnumic  alio  np  roc  ess 
■Mana^menl  raiav 


6.  Resource  Managanenl 

Processes 


Exampks: 

■  Daerminatio  n  and  p  iw  ision 
of  reso  cnees 
■Human  resources 
•Infrastructure  definition 
•  Work  eniTronnteni  definition 


8.  Measuremoit,  Analysis  and 

Inpiwanent  Processes 

Exanqcles: 

•Processes  to  dononstrate: 

•  (  onformitv  of  product 

•  Conformity  of 
management  system 

•  Continual  inpiwanent 


7.  Product  Realization 
Exampks: 

•Planning 

•  Cust> mo1  related  processes 
■Design  &  development  process 
•Puichasingprocess 
•Production  &  serviceprwision 
■  Controlof  monito ring  and 
measuring  dwices 


Figure  6.  Schematic  Representation  of  Typical  Quality  Management  System 
Processes,  related  to  Figure  3.  [From  Ref.  28]. 


Individual  processes  rarely  occur  in  isolation.  Outputs  from  one  process  typically 
form  part  of  the  inputs  into  subsequent  processes,  as  shown  in  Figure  4  (25). 


- ► 

Inputs  to 
Process  A 


Outputs  from 
Process  A 


PROCESS  A 


*■ 


Inputs  to 
Process  B 


PROCESS B 


Outputs  from 
Process  B  p 

Inputs  to 
Process  C 


PROCESS  C 


Outputs  from 
Process  C 

- ► 


Figure  7.  Chain  of  Interrelated  Processes,  [From  Ref.  28]. 

"Continual  improvement"  requires  an  organization  to  focus  on  continually 
increasing  the  effectiveness  and/or  efficiency  of  its  processes,  to  fulfill  its  policies  and 
objectives.  Continual  improvement  (where  "continual"  highlights  that  an  improvement 
process  requires  progressive  consolidation  steps)  responds  to  the  growing  needs  and 
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expectations  of  customers  and  ensures  a  dynamic  evolution  of  the  quality  management 
system  (25,28).  While  the  “continual  improvement”  concept  of  revised  ISO  9001  is 
intended  to  promote  the  effectiveness  of  the  quality  management  system,  the  revised  ISO 
9004  is  intended  to  improve  the  efficiency  of  the  organization.  Together  they  will  help 
increase  organization’s  competitive  advantage  in  the  market,  and  enable  them  to  better 
respond  to  its  customer’s  requirements  (28). 

The  consistency  of  the  two  standards  helps  organizations  that  wish  to  go  beyond 
ISO  9001  to  follow  the  guidance  given  in  ISO  9004.  An  enhanced  requirement  for 
“continual  improvement”  has  been  introduced  into  ISO  9001,  defining  a  complete  cycle 
to  improve  the  effectiveness  of  the  quality  management  system  (48).  Continual 
improvement  is  a  process  of  increasing  the  effectiveness  of  your  organization  to  fulfill 
your  quality  policy  and  your  quality  objectives.  ISO  9001:2000  requires  that  you  plan  and 
manage  the  processes  necessary  for  the  continual  improvement  of  your  quality 
management  system  (52).  Continual  improvement  is  not  an  easy  concept  to  measure. 
Specifically  in  cases  where  product  maturity  has  reached  a  high  level  or  in  many  service 
oriented  purchases. 

A  new  item  that  has  been  introduced  into  ISO  9001:2000  is  the  requirement  for 
the  organization  to  monitor  information  on  customer  satisfaction  as  a  measure  of  system 
performance  (48). "Customer  satisfaction"  is  recognized  as  one  of  the  driving  criteria  for 
any  organization.  In  order  to  evaluate  if  the  product  meets  customer  needs  and 
expectations,  it  is  necessary  to  monitor  the  extent  of  customer  satisfaction.  Improvements 
can  be  made  by  taking  action  to  address  any  identified  issues  and  concerns  (28,48). 
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APPENDIX  G.  ISO  9000:2000  RE-STRUCTURE 


A.  BENEFITS  AND  IMPROVEMENTS 

ISO  states  a  number  of  major  benefits  are  associated  with  the  revised  quality 
management  systems  standards.  Among  them  are  (31): 

•  Applicability  to  all  product  categories,  in  all  sectors  and  to  all  sizes  of 
organizations 

•  Simple  to  use,  clear  in  language,  readily  translatable,  and  easily 
understandable 

•  Significant  reduction  in  the  amount  of  required  documentation 

•  Connection  of  quality  management  systems  to  organizational  processes 

•  Provision  of  a  natural  move  towards  improved  organizational  performance 

•  Greater  orientation  toward  continual  improvement  and  customer 
satisfaction 

•  Compatibility  with  other  management  systems  such  as  ISO  14000 

•  Provision  of  a  consistent  basis  to  address  the  needs  and  interests  of 
organizations  in  specific  sectors  (e.g.,  medical  devices, 
telecommunications,  automotive,  etc) 

•  The  concept  of  the  consistent  pair  -  ISO  9001  covering  the  requirements 
and  ISO  9004  for  going  beyond  the  requirements  in  order  to  further 
improve  the  performance  of  the  organization 

•  Consideration  of  the  needs  of  and  benefits  to  all  interested  parties 

Main  changes  in  the  revised  ISO  9000  standards  are  the  increased  focus  on  top 
management  commitment,  emphasis  on  the  process  approach  within  the  organization, 
continual  improvement  together  with  enhancing  satisfaction  for  customers  and  other 
interested  parties  (48).  The  main  changes  and  new  requirements  that  have  been 
introduced  in  the  consistent  pair  (ISO  9001:2000  and  ISO  9004:2000)  of  quality 
management  system  standards  are: 

The  main  changes  (25): 

•  A  new  process-oriented  structure  and  a  more  logical  sequence  of  the 
contents 

•  A  continual  improvement  process  as  an  important  step  to  enhance  the 
quality  management  system 
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•  Increased  emphasis  on  the  role  of  top  management,  which  includes  its 
commitment  to  the  development  and  improvement  of  the  quality 
management  system,  consideration  of  legal  and  regulatory  requirements, 
and  establishment  of  measurable  objectives  at  relevant  functions  and 
levels 

•  The  concept  of  "Application"  of  the  standard  has  been  introduced  (in 
clause  1.2)  as  a  way  to  cope  with  the  wide  spectrum  of  organizations  and 
activities 

•  A  requirement  for  the  organization  to  monitor  information  on  customer 
satisfaction  as  a  measure  of  system  performance 

•  Significant  reduction  in  the  amount  of  required  documentation 

•  Terminology  changes/improvements  for  easier  interpretation 

•  Increased  compatibility  with  the  environmental  management  system 
standard  ISO  14001 

•  Specific  reference  to  quality  management  principles. 

•  Consideration  of  the  benefits  and  needs  of  all  interested  parties 

•  Addition  of  the  concept  of  organizational  self-assessment  as  a  driver  for 
improvement  (ISO  9004:2000) 

The  main  new  requirements  include  (25): 

•  Continual  improvement 

•  Increased  emphasis  on  the  role  of  top  management 

•  Consideration  of  statutory  and  regulatory  requirements 

•  Establishment  of  measurable  objectives  at  relevant  functions  and  levels 

•  Monitoring  of  information  on  customer  satisfaction  as  a  measure  of 
system  performance 

•  Increased  attention  to  resource  availability. 

•  Determination  of  training  effectiveness 

•  Measurements  extended  to  system,  processes,  and  product 

•  Analysis  of  collected  data  on  the  performance  of  the  quality  management 

system 

The  primary  aim  of  the  "consistent  pair"  is  to  relate  modern  quality  management 
to  the  actual  processes  and  activities  of  an  organization,  including  the  promotion  of 
continual  improvement  and  enhancement  of  customer  satisfaction  (48). 
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B.  ISO  9000:2000  CLAUSES 

ISO  9001  is  organized  under  eight  sections.  The  sections  are  organized  into  eight 
standard  clauses.  They  are: 

Section  1  -  Scope 
Section  2  -  Normative  Reference 
Section  3  -  Terms  and  Definitions 
Section  4  -  Quality  Management  System 
Section  5  -  Management  Responsibility 
Section  6  -  Resource  Management 
Section  7  -  Product  Realization 

Section  8  -  Measurement,  Analysis,  and  Improvement  Of  the  eight 
clause  of  the  new  standard,  Sections  four  through  eight  are  the  most  important  to  the 
QMS.  These  clauses  actually  comprise  the  “heart”  of  quality  management  system.  Each 
one  of  these  five  main  clauses  has  a  number  of  sub  clauses  which  support  the  concept 
identified  in  the  main  clause  (32). 


4.  Quaity  system  requirements 

4.1  Management  responsibility 

4.2  Quaity  system 

4.3  Contract  review 

44  Design  control 

45  Docunert  &  date  Control 

4£Puehasatg 

47  Control  of  Customer  -surpier  Product 

4£  Product  idend-  and  tractability 

4S  Process  centred 

4.10  Inspection  &  testing 

4.11  Ctrl  of  iisp.  mea& lest  quip. 

4,12  Inspection  &  test  results 

4.1 3  Control  of  noncortformitg  pro. 

4.1 4  Corrective  &  preventative  act 

4.1 5  Handing,  storage,  pack,  pres,  &  deft 

4.1 6Controlof  quality  records 

4.1 7  Internal  quaity  audits 

4.18Trabwig 

4.19Servicing 

4.  Quality  Management  System 


5.  Management  Resporrabity 


6.  Resouce  Management 


7.  Product  Realization 


i  Measuemerit,  Analysis  &  friprovements. 
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Figure  8.  Clause  to  Element  Cross  Reference,  [From  Ref.  33]. 

Elements,  clauses,  requirements  are  similar  words  used  in  much  of  the  literature 
produced  on  the  new  standard,  but  they  basically  mean  the  same  thing,  areas  to  be 
addressed.  All  of  the  element  requirements  found  in  the  “1994”  standard  are  reflected  in 
the  “2000.”  This  does  not  mean  that  you  will  find  a  “word  for  word”  transfer  of  the 
requirement  to  the  new  standard.  What  this  means  is  that  the  “concept”  that  was  covered 
has  been  retained,  but  not  necessarily  in  the  same  form  that  was  reflected  in  the  1994 
standard  document.  Please  refer  to  Appendix  B  (ISO  to  MIL-Q  Cross  Reference). 

C.  DOCUMENTATION 

Whole  or  entire  re-structuring  of  an  organization’s  quality  system  or  re-writing  all 
procedures  is  not  expected;  however,  the  revised  standards  include  some  new 
requirements  that  will  need  addressing  in  a  quality  system.  The  three  year  'transition' 
period  allows  accredited  certification  to  the  1994  standards  and  ISO  9001:2000  to 
continue  to  coexist.  This  'transition  period'  will  end  on  15  December  2003.  By  that  date, 
all  organizations  wishing  to  retain  accredited  certification  will  have  to  have  migrated 
their  quality  management  system  to  being  compliant  with  ISO  9001:2000  (25). 

Basically  two  types  of  documentation  exist.  Those  that  tell  us  what  to  do  and 

those  that  tell  others  what  you  have  done  (31).  If  an  organizations  current  quality 

management  system  is  successfully  implemented,  satisfies  the  needs  and  objectives  of  the 

organization,  reflects  the  way  the  organization  works,  addresses  all  of  the  new 
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requirements,  no  changes  are  required.  However,  if  the  current  documented  system  does 
not  address  all  of  the  new  requirements,  additional  documentation  may  be  necessary.  The 
International  Standard  ISO  9001:2000  has  clarified  the  need  for  required  documentation. 
Only  5  documented  procedures,  ISO  9001:2000  Clause  4.2.1,  are  required  by  the 
standards  for  administration  of  the  system;  however,  other  documented  procedures  may 
be  required  by  your  organization  in  order  to  manage  the  processes,  which  are  necessary 
for  the  effective  operation  of  the  quality  management  system.  This  will  clearly  vary 
depending  on  the  size  of  the  organization,  the  kind  of  activities  in  which  it  is  involved 
and  their  complexity  (25).  The  quality  management  system  documentation  includes 
(31,48). 

•  Documented  statements  of  a  quality  policy  and  quality  objectives 

•  Quality  manual 

•  Documented  procedures  required  by  this  International  Standard 

•  Documents  required  by  the  organization  to  ensure  the  effective  planning, 
operation  and  control  of  its  processes 

•  Records  required  by  this  International  Standard 


Figure  9.  Documentation  Pyramid,  [From  Ref.  33]. 
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APPENDIX  H.  INTERVIEWS 


A.  INTRODUCTION 

The  information  presented  in  this  study  was  gathered  through  a  sampling  of  15 
managers,  directors,  or  auditors  of  quality  assurance  management  departments  from 
companies  that  are  currently  ISO  9000:1994  registered  and  the  DCMA.  The  private 
organizations  were  selected  based  on  their  prominent  role  in  the  defense  industrial  base. 
DCMA  representation  was  selected  based  on  their  lead  role  in  organizing  and  distributing 
guidance  and  training  on  ISO  9000  and  expert  knowledge  of  military  quality  assurance 
and  management.  The  aggregate  list  of  respondents  is  presented  in  Appendix  A. 

B.  INDUSTRY  INTERVIEW  RESPONSES 

1.  Question  One  (ISO  9000:2000  Registration) 

Are  you  currently  a  registered  ISO  9000  organization,  and  if  so,  for  how  long? 

a.  Discussion 

The  purpose  of  this  question  was  to  understand  the  background  of  the  interviewee 
and  the  organization  as  it  pertains  to  ISO  Registration. 

b.  Response 

All  of  the  interviewees  were  currently  or  had  in  the  past  been  part  of  ISO 
9000:1994  registration  and  with  DCMA  Audits.  One  interviewee’s  current  department  is 
not  ISO  registered  but  maintains  a  MIL-Q  9858  QMS. 

2.  Question  2  (ISO  9000:2000  Registration) 

Is  the  selection  of  sub -contractors  biased  to  those  registered  ISO  9000  compliant 
and  will  you  require  those  suppliers  to  become  ISO  9000:2000  registered? 

a.  Discussion 

This  question  was  asked  to  query  industry  on  its  supplier  or  sub -contractor  base  in 
an  attempt  to  explore  industry  ability  to  qualify  suppliers  or  vendors  QMS. 
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Reponses 


No,  our  customers  require  that  we  have  a  small  Disadvantaged  businesses 
program  (based  on  a  percentage  of  contract  dollars)  for  procurement  of 
supplies  and  services.  Many  of  these  suppliers  are  family  owned  and  run 
businesses.  The  imposition  of  the  ISO  registration  process  on  these 
suppliers  could  cause  financial/technical  barriers  on  them.  To  alleviate  the 
problem  we  perform  audits  to  insure  that  the  supplier  has  sufficient  control 
in-place  to  insure  that  product  provided  to  GDLS  meets  or  exceed  our 
customer’s  requirements. 

90%  of  our  suppliers  are  ISO  9000  registered.  In  order  for  our  suppliers  to 
maintain  their  registered  status  with  their  registrar  they  must  be  ISO 
9001:2000  by  the  end  of  2003.  The  internal  planning  and  change  process 
to  evolve  to  the  new  standard  is  very  comprehensive  and  driving  the 
requirement  before  the  mandatory  transition  process  is  required  would 
drive  cost  into  programs  that  are  being  squeezed  on  a  yearly  basis  for  cost 
reductions  by  our  customers.  Not  one  of  our  customers  is  willing  to  pay 
more  today  for  improvements  even  with  the  promise  of  future  gains.  With 
the  current  2  to  3  year  cycle  of  Program  Managers  assignments  this  is 
unlikely  to  change  in  the  future. 

Registered  suppliers  avoid  some  of  the  obstacles  required  to  become  part 
of  approved  supplier  base,  however,  registration  is  not  a  pre-requisite. 

No,  it  is  not  biased  in  that  direction.  Our  processes  and  procedures  allow 
for  acceptance  of  a  supplier  based  on  the  systems  and  processes  in  place 
that  will  allow  for  control  of  the  material  manufactured  for  our  customers 
and  us.  In  the  meantime,  the  current  plan  is  to  move  those  suppliers 
currently  on  our  Approved  Supplier  List  (ASL),  as  well  as  those  who  wish 
to  do  business  with  us,  towards  process  control  and  system  administration 
very  similar  to  that  found  in  the  QS  9000  standard. 

We  perform  a  supplier  assessment  based  on  ISO  9000  but  do  not  require 
formal  (3ld  party)  registration.  Although  our  registration  is  now  to  the 
2000  version,  our  supplier  assessment  will  continue  being  performed  to 
the  1994  revision  until  approximately  2003. 

Yes  and  no,  if  they  are  ISO  registered  we  may  not  conduct  a  self- survey  or 
not  as  stringent  of  a  survey,  that  is  the  advantage  to  the  supplier  and  us. 
We  still  deal  with  many  MIL-Q  and  MIL-I  companies  that  have  not 
converted  to  ISO  so  we  may  still  need  to  conduct  other  survey  methods. 
Unless  the  purchase  order  requires  it  we  will  not. 

We  have  established  an  advance  quality  system  with  our  suppliers  based 
on  ISO  9000  and  then  they  are  told  to  comply  with  the  system.  We  do  not 
require  they  receive  3d  Party  registration.  We  will  conduct  a  2d  party 
review.  Past  performance  plays  the  critical  role  in  our  supplier  base. 


•  No  we  do  not  require  ISO  we  do  encourage  it.  We  evaluate  their  quality 
system  approach.  We  look  at  the  quality  manual  and  quality  system,  past 
performance  is  a  big  part  of  it.  We  do  not  stipulate  ISO.  The  advantage  to 
an  ISO  registered  organization  is  we  may  and  often  due  reduce  or 
eliminate  our  periodic  inspection  of  a  supplier. 

•  We  have  no  bias  towards  organizations  that  are  not  ISO.  If  they  are  we 
may  waive  some  of  our  inspections  and  surveillance  but  much  of  that 
depends  on  past  performance  and  the  status  of  the  supplier. 

•  Our  company  uses  a  Quality  Approach  Supplier  Relationship.  It  is 
rigorous  quality  assurance  approach.  A  Quality  System  Review  is  used  to 
identify  potential  suppliers  to  ensure  the  potential  supplier  has  in  place 
appropriate  controls  to  meet  Motorola  standards.  We  do  not  mandate 
suppliers  use  ISO  or  become  ISO  certified  since  our  own  quality  assurance 
provides  greater  detail  and  insight  into  a  supplier.  ISO  does  provide  some 
benefits  but  the  quality  evaluation  is  based  more  on  our  relationship  with 
the  supplier,  past  performance,  length,  and  fulfillment  of  previous 
contracts. 

3.  Question  3  (ISO  9000:2000  Registration) 

What  planning  or  guidance  is  your  organization  conducting  to  meet  the  ISO 
9000:2000  certification  and  what  are  the  significant  changes  it  imposes  upon  your 
organization’s  quality  assurance/management  program? 

a.  Discussion 

This  question  was  designed  to  understand  the  planning  and  guidance 
organizations  are  using  to  meet  the  new  standard  requirements.  The  question  effort  was 
to  gain  a  better  understanding  of  any  common  themes  or  issues  reoccurring  between 
various  industries  as  they  plan  and  execute  ISO  9000:2000  implementation. 

b.  Responses 

•  We  are  currently  reviewing  the  standard  and  will  implement  according  to 
the  guidelines  and  timelines  laid  out  by  our  registrar.  There  are  no 
identified  significant  changes  that  we  will  need  to  address  as  a  result  of 
registration  to  the  new  standards,  although  we  will  be  taking  advantage  of 
the  opportunity  to  more  fully  map  our  processes  and  change  our  work 
instruction  and  procedure  formats.  If  we  find  that  there  is  more  significant 
impact  to  our  company  we  will  find  the  necessary  support  to  secure 
standard  realization. . . 

•  We  established  a  9K2K  (ISO  9001:2000)  transition  team  in  December 

2000,  with  representative  members  from  each  facility  throughout  the 

company.  This  team  was  given  various  forms  of  formal  9K2K  training 
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and  subsequently  developed  our  corporate  ISO  9K2K  transition  plan.  The 
teams  major  transition  compliance  activities  can  be  summarized  in  3 
categories: 

o  Re-write  the  Quality  Manual. 

o  Perform  documentation  gap  analysis  and  revise 
procedures/instructions  as  needed. 

o  Develop  &  Rollout  employee  awareness  and  delta  training. 

Because  of  (what  we  believe  to  be)  a  very  robust  implementation  to  the 
previous  1994  standard,  our  gap  assessments  indicated  that  our 
management  system  was  already  in  compliance  with  the  2000 
requirements.  However,  making  some  modifications  to  existing 
Instructions  to  facilitate  the  availability  of  Objective  Verifiable  Evidence 
for  some  of  the  new  requirements  (e.g.,  Documentation  of  measurable 
improvements)  needed  to  be  made.  Additionally,  significant  effort  needed 
to  be  expended  in  mapping  our  existing  documentation  to  the  requirements 
of  the  2000  standard. 

Yes  we  are  moving  towards  2K  (ISO  9001:2000)  registration,  we  are 
neither  encourage  or  discouraged  from  DoD  to  make  the  conversion  or  to 
push  our  suppliers  to  convert.  There  is  no  significant  or  radical  changes  or 
costs  we  need  to  make  with  our  current  system  to  meet  the  2K 
requirements.  I  do  see  it  as  a  big  change  when  you  compare  the  94 
versions  to  the  2000  version.  For  me  to  explain  how  I  see  the  change  let 
me  provide  a  brief  history  of  ISO  as  I  see  it.  The  87  ISO  emerged  from 
the  European  Common  Market  and  used  ANSI,  MIL-Q,  British  Standards, 
and  ASMI  Code.  So  there  was  nothing  new  about  ISO.  What  was  unique 
about  ISO  was  the  registration  process.  It  was  used  as  a  barrier  for  entry 
into  the  common  market.  The  registration  framework  makes  it  successful 
and  perpetuates  the  success,  financially,  of  the  companies  that  are 
registrars.  It  is  a  product  or  service  the  registrars  provide  for  sale.  ISO  is 
a  commercial  product,  not  a  significant  customer  support. 

Our  transition  is  at  two  levels  at  the  enterprise  and  the  business  unit  level. 
The  company  is  developing  an  overall  enterprise  2k- transition  plan,  but 
each  business  unit  has  their  own  implementation  plan.  We  have  many 
units  at  2k  compliant  already.  We  are  taking  the  lessons  learned  and 
corrective  actions  they  have  made  to  conduct  our  transition  to  2k.  Our 
focus  in  preparing  for  the  transition  is  on  our  business  units  processes 
looks  at  the  corrective  actions  taken  at  other  business  units  and  to  conduct 
on-going  improvements  to  our  processes. 

Our  transition  plan  is  to  meet  the  requirements  by  April  2003.  We  will 
seek  a  joint  registration  between  two  plants.  We  will  focus  our  plan  on 
what  we  see  are  the  significant  changes  in  customer  satisfaction 
measurement,  document  our  continuous  improvement  process,  and  update 
our  quality  manuals.  We  are  looking  at  the  measurement  and  analysis  of 


our  critical  processes  and  goals  connecting  them  to  objectives  and  targets 
for  improvement  and  then  continually  monitoring  that  process  to  continue 
to  improve. 

•  I  see  the  ISO  changes  as  significant.  A  heavy  emphasis  on  continuous 
improvement,  management  oversight,  record  meeting.  It  places  more  a 
burden  on  contractors  to  document  their  continuous  process  improvement. 
I  disagree  that  the  five  disciplines  talked  to  in  the  new  ISO  were  already 
built  into  the  previous  20  elements.  The  IOS  provide  little  guidance  on 
how  to  evaluate  the  new  requirements.  What  is  the  right  process  to 
evaluate  continuous  improvement?  Looking  from  the  quality  systems 
perspective  and  implementing  6  Sigma,  training,  and  resources  in  all  the 
departments  across  an  organization.  ISO  focuses  on  how  to  implement 
and  integrate  all  those  processes  across  you  organization.  The  20  elements 
are  those  things  that  focus  on  product  quality  and  you  cannot  do  with  out. 
The  new  version  includes  those  but  addresses  them  towards  a  global 
implementation. 

4.  Question  4  (ISO  9000:2000  Registration) 

Do  you  believe  the  costs  (financial  and  time  invested)  associated  with  becoming 
ISO  9000:2000  registered  beneficial  (more  profitable  and  better  quality  product)  for  your 
company  and  your  customer  (DoD  and  other)? 

a.  Discussion 

The  cost  of  registration  and  its  benefits  are  an  obvious  start  point  to  discuss  the 
benefits,  if  any,  ISO  9000.  The  researchers  interest  was  to  focus  the  value  added  to  the 
organization,  as  it  would  compare  to  an  organization  implementing  its  own  unique  QMS 
or  the  MIL-Q  standards. 

b.  Responses 

•  I  believe  that  the  implementation  of  ISO  led  us  down  the  current  path  that 

is  in  place  and  that  has  resulted  in  our  company  being  more  competitive 
and  profitable  than  had  we  not  achieved  ISO  certification.  That  said,  I 
believe  that  the  Management  at  my  company  is  committed  to  continuous 
improvement  and  providing  the  highest  quality  product  in  the  marketplace 
while  fulfilling  and  exceeding  the  expectations  of  our  shareholders.  This 
would  be  true  regardless  of  the  Quality  system  we  have  in  place. 

•  Having  previously  been  a  Mil-Q-9858  company,  the  cost  trade  off 

between  ISO  registration  activities  (including  internal  audits  and  other 
internal  efforts)  was  equally  counterbalanced  with  the  absence  of  frequent 
Govemment/customer  quality  system  evaluations. 
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Unfortunately  there  seems  to  be  a  growing  desire  on  the  part  of  our  (non- 
Government)  customers  to  perform  their  own  Quality  Management 
System  audits,  irrespective  of  the  fact  that  an  accredited  3rd  party  registers 
us.  These  audits  are  causing  us  to  evaluate  the  benefit/savings  of  being 
registered. 

The  largest  advantage  that  we  experienced  with  the  ISO  9000  Standard 
(which  ultimately  translate  to  being  more  efficient  thus  more  profitable)  is 
the  flexibility  that  we  (the  supplier)  are  afforded  in  making  our  own 
determination  as  to  HOW  the  requirements  will  be  accomplished.  The 
legacy  Mil- Standard  was  quite  prescriptive  in  not  just  WHAT  but  also 
HOW  requirements  would  be  satisfied,  removing  most  of  the  companies 
ability  to  be  agile. 

ISO  is  not  value  added  to  our  organization.  It  is  more  of  a  marketing  tool 
than  an  improvement  to  our  quality  system.  Our  registrar’s  advice  and 
surveillance  have  been  of  little  use.  Our  business  has  in  use  a 
sophisticated  and  complex  quality  system  and  it  would  be  very  difficult 
for  any  registrar  to  provide  a  great  deal  of  value  towards  our  system. 

The  most  beneficial  outcome  of  ISO  is  to  have  an  outside  registrar  to 
validate  the  quality  cultural  changes  that  need  to  be  made  in  our 
organization.  It  does  not  require  the  quality  department,  often  thought  as 
the  quality  police,  to  champion  the  change.  The  change  required  is 
organization  wide.  It  is  difficult  for  an  organization  to  make  quality 
changes  internally.  ISO  helps  us  focus  on  our  programs  we  want  to 
establish  a  refocus  on  what  we  are  doing  (competencies).  The  1994  ISO 
helped  in  our  process  focus  moving  from  the  previous  MIL-Q.  ISO 
provides  a  competitive  advantage  bit  only  short-term.  Many  organizations 
have  known  for  decades  what  they  needed  to  do  but  were  unable  internally 
to  do  so  because  of  the  culture.  ISO  provided  the  focus  or  catalyst  for 
change. 

Yes  and  no,  before  ISO  we  had  MIL-Q.  There  are  some  differences  but 
there  are  many  common  requirements.  ISO  brought  in  some  other 
standards  like  human  resource  management,  contract  review  and  internal 
audit  process  and  how  to  deploy  those  resources  properly  in  the 
organization.  ISO  expands  your  business  and  defines  your  activities  that 
support  ISO.  The  biggest  benefit  is  the  cultural  change  it  can  bring  about. 
What  we  once  thought  as  unrelated  to  quality  is  actually  related.  It  effects 
the  whole  organization.  Previously  we  looked  at  the  downstream  process 
ISO  brings  in  the  upstream  processes  that  impact  the  product.  Quality 
goes  to  management  commitment,  capable  processes  on  the  manufacturing 
floor.  ISO  can  bring  in  the  discipline  to  change.  I  see  no  significant  costs 
to  change  to  ISO  2000.  They  are  asking  for  things  already  implied  in  the 
1994  standard. 

A  few  years  ago  we  conducted  a  cost  analysis  and  breakout  of  ISO 
registration.  The  largest  barrier  was  in  training.  The  time  it  took  to  train 


the  workforce  and  the  time  it  took  away  from  their  normal  duties.  ISO 
registration  and  preparing  for  it  is  very  human  resource  intensive. 

•  The  ISO  quality  assurance  process  to  follow  ensures  some  consistency, 
and  offers  an  appropriate  methodology  to  use.  The  ISO  registration 
provides  leverage  as  a  sales  marketing  tool  and  to  some  a  competitive 
advantage  although  ISO  is  becoming  more  prevalent  across  industry.  The 
third-party  registration  of  an  unknown  supplier  does  provide  some  benefit 
to  the  buyer  that  some  baseline  of  a  quality  system  exists  in  that  potential 
suppliers  organization.  Some  small  businesses  have  difficulty  in  justifying 
the  cost  to  get  a  third-party  registration.  Third  party  registration  provides 
another  benefit  as  well.  Organizations  often  self  assess  with  ‘blinders’  on; 
they  think  they  are  doing  well  but  may  not  be.  The  third-party  registration 
provides  a  baseline  or  benchmark  to  compare  you  to  other  organization 
and  a  standard  or  template  that  is  industry  accepted  and  proven  to  work  by 
an  “unbiased”  independent  party.  The  cost  of  our  last  registration,  just  the 
visit,  was  $25,000.  This  cost  does  not  measure  numerous  other  resources 
used  for  the  registration. 

5.  Question  5  (ISO  9000:2000  Registration) 

What  are  the  major  barriers  in  becoming  ISO  9000  certified?  Are  there  new 
barriers  introduced  by  the  2000  version? 

a.  Discussion 

This  question’s  aim  was  to  outline  what  impacts  or  obstacles  implementing  ISO 
9000:2000  has  contractually  and  against  resources. 

b.  Responses 

•  The  major  barrier  to  become  certified  is  having  the  required  time  for 
everyone  to  provide  his  or  her  input.  Everyone  in  the  organization  has  his 
or  her  normal  job  functions  and  finding  the  extra  time  can  be  difficult.  I 
don’t  see  any  barriers  in  the  new  revision. 

•  For  a  defense  contractor  the  2000  version  did  not  impose  anything  that  we 
were  not  already  doing.  The  added  requirements  required  us  to  map  those 
activities  to  our  manual. 

•  The  major  barriers  encountered  at  my  business  unit  were  associated  with  a 
new  way  of  looking  at  Quality  as  a  team,  the  “paradigm  shift,”  if  you  will. 
The  increased  emphasis  on  management  involvement  for  the  latest 
standard  will  pose  no  significant  difficulties. 

•  The  initial  barriers  (I  prefer  the  term  challenges)  to  ISO  9000  certification 
are: 
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o  An  initial  and  significant  effort  in  mapping  or  restructuring  one’s 
documentation  system  to  align  with  the  standard. 

o  An  initial  increase  in  the  need  for  employee  awareness  training  due 
to  additional  requirements  (e.g.,  Familiarity  with  the  Quality 
Policy,  Knowing  who  the  ISO  management  Rep  is...)  not 
previously  thought  to  be  necessary. 

o  Establishing  the  budget  for  training  internal  auditors  and  hiring  a 
registrar. 

•  Added  challenges  related  to  the  2000  Standard: 

o  Designing  a  system  to  capture  evidence  of  improvement  programs 
and  upper  management’s  involvement. 

o  Re -calibration  between  us  and  our  registrar  regarding  some  of  the 
new  requirements  and  exactly  what  they  mean.  (Based  on  the  1987 
to  1994  transition,  the  variation  of  interpretation  reduces  with  time; 
but  the  initial  differences  in  opinion  starts  out  fairly  wide) 

o  The  culture  of  us  versus  them,  between  the  quality  inspectors  and 
production,  is  difficult  to  change.  The  culture  change  needed  is  the 
move  to  make  everyone  responsible  for  quality  assurance  in  the 
organization  resulting  in  a  better  product.  Making  each  one  focus 
on  their  processes  in  quality  management  so  we  don’t  have  the 
performance  and  defect  problems  to  begin  with.  The  goal  is  to 
make  sure  all  the  various  partners  in  our  company  take 
responsibility  and  accept  their  function  for  the  quality  of  what  they 
do  whether  it  is  a  process  or  a  product.  This  helps  every  one 
understand  that  quality  is  everyone’s  business. 

•  The  major  barrier  is  cost.  Our  organization  has  no  management  barrier. 

Our  leader  are  versed  in  the  quality  control  process  exceeding  ISO  in 

many  ways,  like  our  6  Sigma  Manufacturing  and  Services. 

6.  Question  6.  (Quality  System  and  Quality  Product) 

One  criticism  of  ISO  9000  registration  is  it  provides  no  guarantee  of  product 
quality  or  conformity,  only  that  a  quality  process  is  in  place.  Do  you  agree,  disagree  with 
this  statement? 

a.  Discussion 

This  series  of  questions  is  based  on  the  recent  incidents  of  ISO  9000  registered 
organizations  producing  non- conforming  and  defective  material.  Its  purpose  was  to  elicit 
comments  on  some  of  the  problems  or  misconceptions  about  ISO  registration  and  what 
measures  can  be  taken  to  increase  quality  control  and  product  conformance. 
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Responses 


No  registration  process  will  guarantee  product  quality.  The  guarantee 
audit  would  be  so  expensive  that  it  would  be  unaffordable,  in  addition, 
there  are  very  few  people  that  could  perform  an  audit  to  the  scope  and 
detail  required  to  verify  every  step  in  every  process  is  acceptable.  Even 
100  percent  inspection  is  only  85%  effective. 

This  is  correct,  however  the  nonconforming  procedures  and  the  corrective 
action  procedures  should  be  finding  the  nonconformance  and  corrects  the 
deficiency. 

I  agree.  A  great  process  must  have  discipline  in  place  to  achieve  the 
desired  results. 

This  would  be  true  of  any  quality  standard.  It  is  not  nearly  as  difficult  to 
achieve  certification,  as  it  is  to  implement  the  underlying  concepts  behind 
the  process. 

I  agree.  ISO  registration  indicates  that  you  have  the  minimum  necessary 
business/management  processes  in  place;  it  is  the  starting  point  for  stellar 
quality  not  the  zenith. 

If  you  really  understand  that  quality  is  everyone’s  responsibility.  If  you 
understand  the  essence  of  the  processes  and  that  is  what  Process  Based 
Management  helps  you  to  do.  The  fact  that  everyone  is  looking  at  their 
processes  through  established  metrics,  and  measuring  the  process  against 
the  metrics,  you  would  be  hard  pressed  to  have  a  major  systems  failure.  I 
see  the  problem  not  with  the  registration  process.  A  company  may  not 
have  bought  into  the  quality  process  they  were  registered  against.  I  am 
speaking  again  of  culture.  You  can  put  the  quality  system  in  place,  get 
registered,  but  if  you  are  not  conducting  the  process  management  or  if 
everyone  is  not  taking  responsibility  for  the  established  process  you  will 
have  problems. 

It  comes  down  to  the  management  philosophy  of  why  you  are  doing  this. 
You  can  fool  the  auditors  but  if  you  use  them  as  a  tool  to  improve  your 
process  you  can  use  that  information  to  make  real  improvements  with  your 
company.  People  of  the  organization  must  embrace  the  need  for  their  role 
in  the  quality  process  of  the  whole  organization  and  its  product.  Key  is  to 
understand  why  we  are  changing  the  culture.  It  is  a  company  generational 
change.  What  I  like  about  ISO  is  how  it  grows  with  the  evolution  of 
quality  standards  and  shifting  culture  over  time.  We  take  it  in  phases.  It 
helps  us  evolve,  our  culture,  management,  and  people. 

I  agree,  no  matter  what  system  you  use  you  could  be  making  good  scrap. 
The  2000  standard  says  you  have  to  have  measurable  targets  or  objectives 
linked  to  continual  improvement.  An  organization’s  policy  objectives, 
cumulative  resources,  and  capable  processes  along  with  ISO  result  in 


quality.  There  is  no  quality  guarantee  with  mere  compliance  to  ISO.  ISO 
provides  a  disciplined  approach  to  maintain  you  system. 

•  If  you  use  the  analogy  of  a  car  air  bag  to  that  of  a  quality  system  like  ISO 
or  MIL-Q,  statistically  you  may  never  use  a  car  air  bag  but  the  time  you  do 
you  are  glad  you  have  it.  A  quality  system  provides  that  check  and 
balance  for  the  time  when  a  quality  issue  or  product  default  presents  itself, 
thus  reducing  the  times  you  create  defective  material.  Quality  addresses 
risk.  Well-defined,  well- implemented  quality  systems  greatly  increase  the 
percent  of  time  you  have  no  defects.  The  quality  system  has  parallels  to 
airport  security.  No  system  can  guarantee  that  no  security  breach  will 
occur  except  one  where  no  one  would  want  to  fly  due  to  all  of  the 
constraints  and  restrictions.  You  want  to  have  a  system  that  provides  the 
greatest  return  on  quality  to  the  cost  of  producing  the  product  to  that 
standard  determined  by  the  level  of  risk  you  are  willing  to  assume.  You 
want  to  get  your  highest  return  on  the  cost  of  the  process,  control 
measures,  and  organizational  resources  you  place  on  a  quality  system. 

•  ISO  9000  is  just  an  audit  of  your  quality  process  providing  a  registration 
that  an  organization  is  meeting  or  following  a  quality  system  in  place,  not 
necessarily  if  the  process  is  being  properly  implemented.  To  some  degree 
ISO  provides  product  quality  assurance  but  there  are  other  intangibles: 
warranty,  consistency,  quest  for  quality,  process  tools  to  continually 
improve,  cost  analysis,  and  product  process  quality  issues  and  resolution. 
No  organization  is  free  from  quality  product  issues.  There  are  always 
concerns  with  suppliers,  safety,  etc.  The  critical  part  is  the  feedback  loop 
and  internal  and  external  to  our  company  to  provide  quality  products. 

c.  Question  6a 

Is  it  possible  for  any  preaward  survey  of  quality  assurance  to  detect  indicators  of  a 
sub- standard  product  or  service? 

•  Yes,  remember  that  the  ISO  9001:2000  standard  is  a  Quality  Management 
System.  You  also  need  to  know  that  there  were  many  debates  about 
taking  the  "Quality"  out  of  the  title  during  the  development  process. 

•  No,  on  a  small  insignificant  matter.  Yes,  if  the  system  has  a  major 
breakdown. 

•  No,  a  preaward  survey  will  only  indicate  the  presence  of  capability,  and 
will  not  provide  assurance  of  the  actual  product/service  quality 

•  The  preaward  survey  can  provide  a  companies  core  competencies  and 
capabilities.  It  can  detect  a  critically  sub-standard  quality.  Will  it  detect  a 
potential  bad  product  or  service?  I  say  it  depends  on  the  qualifications, 
skills,  and  insights  of  the  auditors  more  than  any  survey  or  checklist. 
DCMA,  at  best  looks  at  an  organization  in-house,  for  two  days  for  a 
preaward  survey,  it  can  only  provide  a  good  basic  feel. 
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d.  Question  6b 


What  guarantees  does  ISO  9000  certification  provide  towards  a  quality  achieving 
organization  resulting  in  a  quality  product? 

•  The  certification  is  not  the  key  internal  process  management  and  controls. 
The  management  system  including  internal  audits  are  key  to  an 
organization  achieving  a  quality  product.  We  have  averaged  about  .5 
minor  nonconformances  across  all  of  our  facilities  and  all  audits  since 
1997. 

•  Any  organization  can  put  together  a  system  that  can  attain  registration;  the 
key  is  the  desire  to  perform  at  half  throttle  or  at  full  throttle.  The  greater 
the  effort  is,  the  greater  the  return. 

•  None,  I  do  not  believe  that  ISO  9000  is  a  guarantees  quality;  rather  it  is  the 
single  best  indicator  of  the  ability  to  produce  a  quality  product. 

•  ISO  procedures  are  not  intuitive  and  are  not  directly  related  to  the  product. 
ISO  procedures  do  not  yield  a  product;  good  procedures  or  good 
requirements  do  not  necessarily  yield  a  good  product.  It  is  more  directly 
related  to  the  process.  We  have  systems  audits,  business  controls,  ISO, 
manufacturing  techniques,  SPC,  etc.  The  degree  of  compliance  is 
measured.  Do  we  have  perfect  compliance  to  our  process  procedures,  and 
if  we  do  have  perfect  compliance  then  do  we  have  a  good  product.  For 
example,  do  we  have  a  good  operation  but  the  patent  dies.  The  true  test  of 
process  procedures  is  not  if  they  meet  some  artificial  requirement  like 
ISO.  The  test  is,  “Does  the  product  work!”  If  you  have  processes  and 
procedures  that  produce  good  product  then  you  have  something.  You  then 
establish  those  procedures  in  writing  and  then  train  to  them  and  measure 
them.  The  existing  procedures  become  experience  retention  in  your 
workforce.  Experience  retention  and  experience  culture  make  a  good 
organization.  This  experience  is  put  back  into  your  continuous  process 
improvement  and  control. 

e.  Question  6c 

What  value  added  does  the  ISO  Registration  have? 

•  A  second  set  of  eyes,  a  regularly  scheduled  review  that  forces  you  to  put 
things  in  order  twice  a  year.  And  the  cost  is  a  management  commitment  to 
support  the  management  system. 

•  Our  registration  helps  to  maintain  our  customers’  confidence  and  that  of 
the  public. 

•  It  sets  a  direction  that  is  up  to  the  team  who  implemented  and  achieved  the 
certification  to  understand,  and  allows  for  the  base  lining  of  improvements 
achieved  through  continuous  improvement  efforts. 
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•  When  recognized  by  the  customer,  ISO  registration  eliminates  the  need  for 
customer  assessments;  yielding  instead  to  certified  3  party  audits.  This  is 
a  saving  to  the  customer  and  to  the  contractor  in  that  a  single  registration 
audit  can  eliminate  multiple  customers  performing  similar  duplicated 
assessments. 

•  An  ISO  9000  registration  is  commonly  understood  across  the  industry. 
Where  in  the  past  a  thesis  was  expected  to  articulate  your  quality  system; 
now  simply  showing  ISO  registration  is  understood  and  accepted.  I  equate 
ISO  9000  registration  to  a  college  education.  In  the  same  sense  that  a 
college  degree  neither  means  that  the  graduate  is  ‘smart’  nor  does  it  mean 
that  the  graduate  can  function  as  an  experienced  practitioner  in  his/her 
field;  the  education  does  demonstrate  that  the  student  has  successfully 
demonstrated  the  ability  to  learn  and  possesses  the  basic  understanding  of 
elements  of  the  field  that  he/she  has  studied. 

•  Similarly,  ISO  9000  does  not  mean  that  a  company  produces  a  great 
product;  but  has  demonstrated  that  it  possesses  all  of  the  basic  processes 
deemed  necessary  in  order  to  produce  a  quality  product/service. 

•  The  value  added  is  the  industry  wide  recognition  of  the  standard. 

7.  Question  7.  (Quality  System  and  Quality  Product) 

What  quality  assurance/management  measures  do  you  take  to  insure  your  own 
process  and  that  of  your  suppliers  demonstrates  product  conformity  and/or  critical 
characteristics,  in  terms  of  defect  levels,  are  acceptable;  and  that  improvements  are 
planned  and  later  demonstrated? 

a.  Discussion 

This  question  was  asked  to  generate  responses  to  address  issues  raised  in  the 
previous  question  on  how  to  address  quality  assurance  above  and  in  addition  to  ISO. 

b.  Responses 

•  We  use  trend  analysis  to  show  improvements. 

•  We  have  a  broad  range  of  measurements  from  sampling,  SPC  100% 
inspection,  etc. 

•  Internal  and  external  audits  are  the  most  common  means  of  system  and 
process  validation,  although  a  sampling  of  material  is  also  a  must.  We 
also  monitor  material  at  all  phases  of  manufacture  and  note  any 
deficiencies  and  associate  those  deficiencies  with  a  responsible  party 
through  root  cause  analysis.  Corrective  actions  are  reviewed  and  accepted 
and  validated  in  each  case  where  failure  has  occurred. 
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Product  evaluations  are  the  major  “added”  activity  to  those  already 
addressed  by  the  ISO  standard  which  ensure  product  conformity, 
critical/key  characteristic  compliance  and  that  the  deliverable  product  is  a 
quality  item. 

We  use  many  tools,  SPC,  ISO  as  a  baseline  in  some  areas,  varying  trend 
analysis  etc.  The  key  to  success  is  management’s  ability  to  know  when  to 
use  the  right  tool. 

We  have  advanced  quality  procedures  levied  against  our  suppliers.  They 
focus  in  three  areas,  variability  control,  hardware  quality  audits,  and  ISO 
process  management  approach.  Internally  we  have  a  constant  source  of 
metrics  that  we  manage  with  DCMA,  internal  management,  suppliers  and 
each  business  function.  We  have  transitioned  the  metrics  from  a  product 
focus  where  we  tracked  results  and  defects  to  more  up  front  in  the  process 
measuring  something  that  will  not  allow  us  to  have  the  defect. 

We  use  our  supplier  evaluation  selection  process.  It  measures  and 
monitors  suppliers  processes  and  controls,  involves  a  supplier  rating 
system  and  provides  procurement  quality  assurance.  It  monitors  our 
suppliers  through  on-site  inspection  and  upon  receiving  a  product.  The 
evaluation  or  rate  dictates  the  amount  of  surveillance  and  oversight  we 
use. 

Other  measures  have  been  in  developing  a  supplier  corrective  action 
system  where  we  have  partnered  with  our  suppliers  to  develop  a  better 
quality  product.  We  have  reduced  our  supplier  base  to  gain  better  control 
and  we  hold  an  annual  supplier  symposium  over  three  days  to  discuss 
future  business  plans  and  operations. 

In-house  we  use  non-compliance  control  corrective  action.  We  have 
quality  improvement  boards  for  nearly  every  business  function  and  unit. 
These  boards  use  the  experience  and  information  from  the  people  actually 
implementing  the  process  to  make  improvements.  Information  and 
improvements  from  our  quality  improvement  boards  that  meet  weekly  and 
our  quality  improvement  council  that  meets  quarterly  involve  all 
stakeholders  to  improve  our  organizations  quality  improvement  process. 

The  key  is  to  have  a  flexible  and  dynamic  system,  whether  it  is  ISO  or 
MIL-Q,  that  responds  to  the  conditions  at  hand.  The  system  needs  a  list  of 
menu  options  or  courses  of  action  to  follow  to  implement  to  distinguish 
between  a  trend  and  an  isolated  incident  with  a  vendor. 

We  have  many  pre-ship  quality  processes.  Most  are  based  on  the  CMM 
Model  and  follow  CMI  and  our  Standard  Processes.  We  have  a  process 
that  looks  at  and  measure  the  critical  paths  of  a  product;  design,  build, 
manufacturing,  test  and  function,  focusing  on  the  products  “critical 
characteristics.”  We  also  use  break  and  fix,  accelerated  life  testing  to 
mature  our  products  to  measure  quality  level. 


•  On  the  production  and  service  side  we  have  numerous  management 
review,  various  avenues  for  customer  feedback,  tracking  systems  like 
systems  defect  trends.  All  of  these  procedures  are  linked  to  our 
organization’s  Failure  Review  Board  involving  all  product  players 
(stakeholders).  Suppliers  and  customers  are  a  part  of  the  entire  process 
and  manufacturing  relationship.  An  output  is  our  Quality  System  Report 
Card  providing  our  process  to  improve. 

8.  Question  8.  (Quality  System  and  Quality  Product) 

Have  the  problems,  shortcomings,  and  lessons  learned  with  the  previous  ISO 
9000  standards  and  registration  been  addressed  under  ISO  9000:2000  and  have  the  latest 
technologies  and  best  business  practices  been  introduced  into  the  new  ISO  standard?  If 
not,  what  are  your  recommendations  on  what  should  be  addressed  and  introduced? 

a.  Discussion 

This  question  was  used  to  allow  organizations  to  provide  the  researcher  with 
insight  into  what  quality  assurance  practitioners  saw  as  the  strengths,  weaknesses,  and 
opportunities  of  both  ISO  9000:1994  and  2000  provide. 

b.  Responses 

•  They  did  a  great  job.  Could  there  be  improvements?  Always!  Getting 

consensus  from  100  countries  and  thousands  of  representatives  is  near 

impossible.  Let  the  evolution  to  this  standard  happen,  I  think  we  will  find 
that  the  Registrars  and  Companies  will  incorporate  the  standard  in  the 
spirit  of  it's  intent 

•  The  document  (ISO  9000:2000)  is  biased  towards  the  supplier  and  what  it 
takes  to  meet  the  document.  Suppliers  have  the  most  influence  on  the 
document.  It  is  tailored  for  them.  What  are  the  buyers  risk  when  they 
purchase  something?  The  buyer  community  has  not  been  addressed  in  the 
2000  version.  The  customers  of  the  registrars  influenced  ISO  2000.  I  see 
it  as  a  less  effective  document  and  aimed  more  towards  the  service 
industry  rather  than  manufacturing. 

•  No,  our  requirements  always  have  and  always  will  be  more  stringent  than 
commercial  industry,  not  to  mention  we  have  our  customer  living  within 
our  facility. 

•  In  my  opinion,  the  new  standard  has  complicated  process.  The  new 
standard  will  create  a  reduction  in  registration  of  small  companies  (500  or 
less)  due  to  the  additional  requirements,  which  will  increase  business  cost. 

•  I  do  not  believe  a  standard  should  dictate  technologies  and  business 
practices  but  rather  should  provide  a  framework  for  Management  to 
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determine  what  works  best  for  the  particular  product  manufactured  and 
sold.  The  previous  ISO  standard  built  the  foundation,  and  it  was  ample. 
The  latest  revisions  appear  to  require  added  management  involvement  and 
recognize  the  cross- functional  requirements  that  are  necessary  in  an 
optimally  running  company.  What  has  occurred  in  my  eyes  is  the  standard 
has  grown  to  better  reflect  the  current  business  environment. 

The  only  improvement  that  I  see  in  the  2000  version  is  that  it  is 
restructured  to  eliminate  the  perception  that  ISO  9000  is  20  separate,  stand 
alone  elements  that  “the  quality  guy”  is  responsible  to  ensure  are  met. 
Unfortunately,  a  large  degree  of  vagueness  in  the  requirements  of  the 
standard  accompanies  the  new  revision. 

ISO  2000  is  a  significant  and  radical  departure  and  move  towards  supplier 
ease  of  implementation  as  opposed  to  effectiveness  as  a  customer  tool. 
The  new  version  is  more  concerned  with  producer  risk  than  consumer  risk. 
The  ISO  standard  is  neither  good  or  bad,  it  is  not  a  silver  bullet  of  quality 
to  an  organization  either.  I  would  not  give  the  ISO  document  to  a  new 
company  to  begin  business.  It  is  not  a  specification  but  more  a  course  in 
general  management.  I  see  three  new  areas  in  ISO  2k,  continuous  process 
improvement,  quality  objectives,  and  customer  satisfaction.  While  these 
are  new  to  the  document  does  it  help  be  make  better  widgets?  The  whole 
focus  of  the  three  is  nothing  new  any  good  business  is  doing  these  things 
already  or  they  would  not  be  or  continue  to  be  in  business.  The  document 
forces  customers  to  provide  feedback.  Why  should  ISO  force  feedback? 
Every  time  a  customer  returns  to  me  they  are  providing  feedback.  An 
organization  cannot  remain  in  business  if  they  do  not  receive  feedback. 
The  document  should  be  intuitive  for  any  competitive  market  in  a 
capitalistic  society.  ISO  provides  no  edge  it  is  generic  it  wants  everyone 
to  be  gray.  It  lends  itself  better  to  socialistic  markets  where  competitive 
prices  of  the  market  place  are  not  allowed  to  establish  the  market.  Our 
markets  (U.S)  require  organizations  to  have  an  edge  or  advantage  to 
prosper.  ISO  provides  only  a  baseline  and  should  not  be  used  to 
distinguish  between  what  an  organization  is  to  use  and  how  it  will  provide 
the  best  product  or  service.  Do  we  want  a  level  playing  field  of  quality 
management?  Should  every  one  be  on  an  equal  basis  or  in  a  standard 
way?  Our  ISO  registrar  criticized  the  way  we  managed  some  areas  stating 
it  was  not  like  everyone  else  did  it  or  as  the  ISO  document  prescribes.  My 
reply  is,  “Exactly  that  is  what  provides  us  an  edge  over  our  competitors 
and  provides  us  more  of  a  market  share  and/or  return  on  investment.”  It  is 
our  quality  management  secret  and  strategic  advantage;  do  not  give  it  to 
other  companies  you  registrar.  I  would  not  recommend  that  a  company 
follow  ISO  and  nothing  else  because  it  leaves  them  equal  with  every  one 
else.  Businesses  succeed  or  fail  on  their  ability  to  manage.  There  is  no 
quality;  quality  management  is  an  illusion  invented  by  academics  and 
registrars  who  want  to  create  a  discipline  where  one  does  not  exist  it  is  an 


art.  An  organization  and  its  people  must  be  competent,  capable, 
responsible,  and  accountable  for  doing  the  right  thing. 

•  The  major  changes  we  see  in  the  ISO  2k  is  its  focus  on  process-based 
management.  For  our  organization  the  program  is  to  identify  our  critical 
processes,  develop  a  plan,  and  use  our  seven-step  process  to  evaluate  the 
process  and  improve  it  through  the  monitoring  process.  Our  seven-step 
process  of  development  fits  well  into  the  ISO  standard  and  its  process 
based  management  approach.  Through  understanding  your  process  and 
developing  metrics  around  that  process  the  measure  how  you  are 
performing  is  how  to  continue  improvement  in  the  process.  Previously 
ISO  was  a  system  that  looked  at  content  and  processes  based  on  a  process 
now  ISO  focuses  on  process  quality  ownership. 

•  There  are  explicit  and  implied  changes  in  the  new  ISO  most  are  adapted 
from  the  process  business  management  approach  that  asks  for  an 
organization  to  define  its  core  competencies,  relationships  between  each 
process,  what  the  process  flow  is,  and  the  criteria  if  capable  or  not.  I  see  it 
failed  to  address  much  on  safety.  It  does  look  at  the  work  environment  but 
many  define  that  very  narrowly. 

•  I  do  not  see  many  shortcomings.  What  is  obvious  to  some  in  the  previous 
20  elements  is  not  so  to  others.  What  organizations  see  as  shortcomings  or 
weaknesses  and  even  strengths  of  ISO  is  largely  dependent  on  the  culture 
of  the  organization  implementing  ISO.  It  is  largely  dependent  on  the  level 
of  importance  and  role  culture  plays  in  allowing  the  organization  to 
understand  ISO  techniques  of  quality  assurance  and  management 
resolution  to  champion  the  organizational  change  to  the  current  culture. 

9.  Question  9.  (Quality  System  and  Quality  Product) 

Will  ISO  9000:2000  emphases  on  customer  focus,  product  realization  and 
conformity,  process  management,  and  resource  management  address  the  past  shortfalls 
associated  with  the  1994  ISO  version? 

a.  Discussion 

The  question  was  designed  as  a  follow  up  to  the  previous  question  to  address  the 
effectiveness  of  some  of  the  new  changes  built  into  ISO  9000:2000. 

b.  Responses 

•  Yes,  it  will  help  to  cause  continuous  improvement  in  those  and  other 

areas. 

•  No,  I  believe  that  the  attempt  to  add  these  emphases  added  a  greater  lack 
of  clarity.  All  of  these  ‘elements’  existed  in  the  previous  version. 
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•  I  have  three  main  criticisms  of  the  new  ISO  document.  One  is  its  focus  on 
producer  risk  rather  than  customer  risk.  The  first  three  section  of  the 
document  (ISO  9001:2000)  are  confusing  and  unclear  it  is  not  until  you 
get  to  section  four  that  you  find  any  substance.  The  dilution  of  the  20 
elements  into  the  new  five  area  of  quality  management  is  a  Harvard 
Business  School  course  on  how  to  manage,  not  a  good  quality  system. 
The  1994  version  provides  a  checklist  of  the  basic  fundamentals  of 
management;  however,  in  the  2000  version  all  of  that  is  gone.  The  1994 
version  provided  a  quick  way,  or  a  rough-cut  or  view,  for  customer 
auditors  to  evaluate  suppliers.  The  new  standard  is  deplorable  because  it 
is  harder  to  determine  what  it  is  requiring. 

•  The  previous  20  elements  are  now  spread  against  the  new  five  disciplines 
plays  well  into  how  key  stakeholders  in  quality  management  development 
are  reorganizing.  It  moves  away  from  the  stovepipe  approach  of 
organizations  where  one  or  some  function  was  responsible  for  one  or  a 
series  of  the  elements.  ISO  now  spreads  the  previous  20  elements 
responsibility  across  the  entire  organization  using  the  five  disciplines. 
You  cannot  do  away  with  the  20  elements;  they  are  the  basics,  the 
premises  of  business.  ISO  did  not  invent  a  total  new  way  of  doing 
business;  just  new  ways  to  better  develop  your  own  processes. 

•  My  concern  with  the  2000  version  is  the  customer  satisfaction 
measurement  as  it  applies  to  organizations  that  primarily  deal  with  the 
DoD  as  a  customer.  Customer  satisfaction  is  not  an  easy  thing  to  measure 
when  DoD  is  >our  customer.  We  have  many  customers,  DCMA,  DCAA, 
the  buying  command,  the  program  office,  the  systems  manager,  and 
ultimately  the  soldier.  ISO  does  not  tell  you  how  to  do  it  just  that  some 
process  needs  to  be  in  place  to  estimate  it. 

10.  Question  10  (DoD  and  Contractor  Quality  Assurance  Relationship) 

Has  ISO  9000  become  the  quality  management  standard  for  DoD  contractors,  i.e., 
must  a  contractor  become  ISO  9000  certified  to  conduct  business  with  the  DoD 
especially  in  contracts  requiring  a  higher- level  quality  standard? 

a.  Discussion 

The  issue  the  researcher  wanted  to  raise  in  this  question  is,  although  ISO  is  not 
mandated  by  any  Federal  procurement  regulation  and  guidance,  has  the  ISO  influence 
created  an  environment  in  which  industry  must  be  compliant  with  ISO  to  conduct 
business. 


Ill 


Responses 


Not  that  I  am  aware  of.  Nor  should  it  be.  From  a  competitive  standpoint 
the  DoD  should  compete  its  contracts,  as  any  good  business  would  do.  It 
should  insure  that  the  material  it  requires  meets  its  requirements  and  that 
the  place  of  manufacture  is  consistent  in  its  performance.  Some 
companies  are  quite  capable  without  certification  to  meet  any  customer’s 
standards  and  expectations.  If  all  contractors  were  required  to  be  ISO, 
would  that  eliminate  those  who  are  registered  to  other  standards,  QS  9000, 
for  example?  Being  certified  and  maintaining  that  certification  through 
semi-annual  audits  helps  to  mitigate  risk  to  the  customer,  and  the  DoD  is 
after  all  a  customer.  A  company’s  commitment  to  high  quality  is  not 
simply  reflected  in  its  registration.  It  is  reflected  in  its  workforce  and 
product. 

What  concerns  me  is  the  use  of  ISO  9000:2000  in  contracts  and  to  qualify 
my  organization  as  a  supplier  under  that  standard,  especially  the  measure 
and  aspect  of  continuous  process  improvement  and  product  realization. 
Those  are  both  difficult  to  measure  contractually.  For  example,  lets  say 
you  manufacture  bicycles  and  you  have  made  bicycles  for  years.  You 
initially  improved  from  90%  to  95%  then  95%  to  99%.  Know  you  are  at 
the  99.99%  you  will  have  little,  if  any,  improvement.  Does  that  mean  I 
failed  to  meet  my  contractual  obligations?  Will  I  be  terminated  for  not 
improving  my  product?  Improvement  is  a  good  thing  but  not  an  easy 
contractual  element  to  accurately  measure.  The  key  is  to  provide 
motivation  for  the  supplier  to  improve  on  the  contract  not  to  punish  the 
supplier  if  they  do  not.  Motivated  suppliers  to  improve  and  provide  good 
products.  Contract  risk  is  mitigated  through  the  motivating  the  supplier 
through  contract. 

Some  would  say  yes,  the  way  contracts,  statements  of  work  are  written  it 
has  become  a  de  facto  requirement.  Of  course  it  is  always  dependent  on 
the  contract  and  what  an  organization  is  using  the  contract  to  purchase. 

No,  it  can  be  ISO  or  equivalent.  MIL-Q,  QS9000  that  is  more  restrictive 
and  prescriptive  can  also  be  used. 

No,  in  fact  in  November  of  1999  we  participated  in  a  program  to  update 
MIL-Q  9858  checklist  and  be  evaluated  against  the  revised  checklist.  The 
reason  DCMA  wanted  to  revise  the  past  checklist  was  the  realization  that 
many  DoD  contractors  wanted  to  remain  and  still  use  the  MIL-Q  standards 
and  did  want  to  move  to  ISO.  Many  organizations  were  and  continue  to 
work  well  under  the  MIL-Q  standards  and  could  not  justify  the  cost  to 
switch  to  a  new  system.  MIL-Q  is  more  directives  than  ISO  is  but  ISO 
contains  many  “shalls”  and  “wills”. 


11.  Question  11.  (DoD  and  Contractor  Quality  Assurance  Relationship) 

Should  DoD  impose  ISO  9000  standards  and  registration  on  contractors  and  sub¬ 

contractors  as  an  entrance  to  conducting  business  with  the  DoD? 

a.  Discussion 

This  question  was  meant  to  elicit  general  comments  on  the  benefits  and  problems 
of  a  DoD  mandate  of  ISO  9000. 

b.  Responses 

•  No,  this  puts  a  financial  burden  on  small  companies. 

•  All  contractors  have  a  system  that  meets  or  exceeds  this  due  to  the  nature 
of  our  business  after  all  ISO  is  an  offspring  of  9858.  If  the  DOD  were  to 
impose  registration  then  defense  contractors  would  be  able  to  bill  the 
DOD  for  our  ISO  registration  cost.  This  will  not  happen. 

•  Yes,  but  if  3ld  party  registration  is  imposed,  DOD  audits  should  be 
removed. 

•  In  my  opinion,  DoD  cannot  use  the  blanket  policy  of  choosing  only  ISO 
registered  organizations  to  do  business  with.  Commercial  buyers  and 
producers  must  be  better  than  that  (ISO).  Selection  must  be  process  and 
skill  determined  and  situation  dependent.  You  want  to  use  source 
inspection  and  performance  data  and  feed  that  into  a  process  of  source 
selection.  DCMA  and  DoD  has  not  done  a  good  job  of  doing  that  they 
have  rot  invented  the  proper  tools  to  use  to  make  that  process  work  as 
well  as  industry  has.  A  good  example  of  how  this  works  is  a  vendor¬ 
rating  program.  Product  is  inspected;  performance  data  is  generated  and 
processed  into  our  vendor-rating  program  across  the  organization.  We  use 
it  as  a  tool  for  future  supplier  selection.  I  see  DCMA  request  the  data  but  I 
don’t  see  a  documented  transfer  or  process  to  provide  the  information 
across  DoD  organizations. 

•  In  general  terms  no,  but  ISO  has  influences  on  all  of  the  se. 

•  Maybe,  DoD  should  look  at  whether  the  standard  meets  what  they  are 
buying.  Most  contracts  say  ISO  or  equivalent,  most  of  our  international 
contracts  specify  ISO  by  name.  With  our  international  business  we  must 
have  ISO. 

12.  Question  12.  (DoD  and  Contractor  Quality  Assurance  Relationship) 

Some  DoD  Quality  Assurance  Representatives  (QARs)  complain  some  ISO 
registered  companies  believe  that  if  they  are  ISO  registered  that  only  limited  DoD 
inspection  is  required.  Should  DoD  QARs  accept  ISO  registration  as  certification  for  the 
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Preaward  Quality  Assurance  Survey  and  has  registration  reduced  and/or  eliminated  DoD 
on-site  quality  inspections  or  audits  during  the  surveillance  phase,  should  it? 

a.  Discussion 

This  question  was  posed  to  address  one  of  the  major  problems  associated  between 
the  DCMA,  industry,  and  registrars  of  ISO.  The  questions  goal  was  to  bring  into  context 
the  Third-party  registrar  and  DCMA  role  in  quality  assurance  audit  and  surveillance. 

b.  Responses 

•  I  agree  the  Preawards  could  be  eliminated,  and  yes  the  on  site  quality 

inspections  should  be  reduced. 

•  The  surveillance  audits  are  quick  passovers  to  check  for  system 
breakdowns.  To  fully  evaluate  a  system  as  critical  as  ours  requires  detailed 
surveillance.  We  have  a  unique  system  in  that  our  customer  participates  on 
our  monthly  Internal  Audit  Reviews  thereby  maintaining  oversight 
without  performing  additional  disruptive  activity. 

•  Should  DoD  QARs  accept  ISO  for  the  preaward?  Since  preaward  implies 

no  final  selection  has  been  made,  yes.  After  a  preaward  is  conducted  (for 
example,  a  desk  audit)  and  before  signing  contracts,  it  is  always  good 
business  to  visit  the  site  and  ascertain  its  true  capabilities.  That  would  be 
a  pre-contracting  audit  and  should  be  performed. 

•  Has  ISO  certification  resulted  in  a  reduction  of  on-site  customer  audit 
personnel?  In  our  case,  no.  It  should  be,  though.  As  a  company  matures 
into  the  ideals  surrounding  ISO  and  continuous  improvement  efforts  prove 
to  reduce  the  risk  to  the  customer  well  below  the  noise  level  it  would  seem 
to  be  a  waste  of  money  to  maintain  large,  on-site  oversight  personnel. 

•  The  areas  (or  elements)  assessed  by  the  3  party  should  not  be  re-audited 
by  DOD.  However  many  DOD  evaluations  go  beyond  the  ISO 
requirements.  ...DOD  should  evaluate  the  delta;  otherwise  an  added 
burden  has  been  placed  on  the  contractor. 

•  I  do  not  think  it  should  be  based  on  ISO  registration.  We  should  be  able  to 
prove  to  our  customer  the  quality  of  the  product.  We  should  be  able  to 
conduct  an  adequate  job  of  monitoring  and  continually  improving  the 
measurement  of  our  process,  whether  or  not  we  use  ISO,  MIL-Q  or  any 
other  system,  that  should  be  their  (DCMA)  justification  for  backing  off  on 
the  amount  of  oversight  and  surveillance.  The  system  is  not  as  important 
as  how  we  measure  up  to  our  customer’s  expectations  and  requirements. 
It  is  a  matter  of  trust  based  on  data. 

•  The  amount  of  surveillance  or  oversight  should  not  be  based  on  any 
redundancies  found  between  registrars  and  DCMA  but  driven  more  by 
how  well  the  process  is  working  and  measuring  up  to  the  oversight  DCMA 
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feels  they  need  to  conduct  and  as  to  how  we  are  evaluated  to  contractual 
performance  measures.  The  relationship,  if  one  exists,  is  if  ISO  has 
helped  us  improve  and  organize  our  process  to  a  point  to  where  DCMA  is 
more  confident,  not  the  ISO  registration.  It  is  a  matter  of  confidence  and 
risk  associated  with  the  contract  not  a  correlation  of  registrars  and  DCMA. 

•  I  don’t  think  ISO  alone  has  reduced  inspectional  and  oversight.  I  believe 
low-budgets,  past-performance,  fewer  resources,  ISO,  and  our  good 
relationship  with  DCMA  has  reduced  surveillance  and  inspection. 
PROCAS  and  our  partnering  with  DoD  have  also  reduced  the  amount  but 
not  ISO  alone. 

•  There  are  differences  among  registrars  and  differences  among  DCMA 
auditors.  Go  vemment  representatives  come  from  different  agencies,  from 
various  areas  and  communities,  with  varying  objectives  against  the  same 
contract.  DoD  agencies  waste  time  with  the  infighting  that  occurs 
between  the  various  DoD  stakeholders.  It  constrains  DCMA  and  all  the 
other  DoD  agencies  to  adequately  evaluate  an  organization  because  DoD 
cannot  reach  consensus  on  what  they  expect  of  the  contractor  performance 
against  the  contract.  The  roles  of  contract  administration  are  not  agreed 
upon  between  DCMA  and  the  buyer. 

•  The  PCO  and  ACO  must  clearly  designated  and  define  the  areas  of 
contract  administration  and  procurement  functions.  The  resident  DCMA 
representatives  should  take  the  lead  in  deciding  the  jurisdiction  of  the  PCO 
and  the  ACO  in  many  of  these  matters. 

13.  Question  13.  (DoD  and  Contractor  Quality  Assurance  Relationship) 

How  should  the  preaward  survey  and  the  follow-on  quality  surveillance  of  DoD 
QARs  be  conducted  (consider  training,  DoD  culture  changes,  and  in-plant  conduct)? 

a.  Discussion 

This  question  was  asked  as  a  follow  up  to  the  previous  question  to  solicit 
comments  on  how  best,  in  industries’  view,  to  address  issues  raised  in  Question  11. 

b.  Responses 

•  If  there  is  a  preaward  focus  on  the  performance  spec  of  the  product  not  the 
quality  system.  QARs  should  be  invited  to  the  management  review 
meetings  to  understand  how  the  quality  system  is  performing  and  this 
would  be  an  opportunity  to  provide  customer  input. 

•  You  establish  the  expectations  before  award.  Validate  a  firm 

understanding  of  how  you,  as  the  customer,  expect  things  to  be  done. 
Validate  that  those  materials  being  purchased  can  be  assembled  or 
manufactured  IAW  the  requirements.  Validate  the  beginning  of 
production  through  thorough  audits  of  the  processes.  Then  establish  the 
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plan  for  follow  on  visits,  while  letting  the  supplier  know  that  any  issues 
that  arise  will  bring  your  group  in  as  a  reactionary  force  to  assist  with 
issue  resolution.  While  this  relationship  is  being  established,  you  must 
also  express  that  you  have  a  shared  interest  in  the  success  of  the  program 
and  are  willing  to  do  whatever  is  necessary  to  insure  that  success. 

•  I  see  no  change  in  the  procurement  assessment  process  other  than  using 
(and  not  duplicating)  the  ISO  registration  as  an  accredited  certificate  that 
the  Quality  Management  System  is  acceptable.  This  should  not  affect 
other  parameters  of  pre/post  award  assessment  of  suppliers. 

•  It  depends  on  how  DoD  chooses  to  use  DCMA.  DCMA  are  not  registrars 
and  registrars  are  not  DCMA.  Registrars  are  system  oriented  and  conduct 
a  paper  work  audit  where  DCMA  is  more  hardware  or  product  oriented 
and  conduct  audits  to  that  degree.  Both  organizations  depend  a  great 
degree  on  the  skills  and  knowledge  of  the  person  and  not  so  much  the 
organizations  mandate.  It  is  difficult  to  find  people  with  both  the  skills 
and  ability  to  look  at  both  systems  approach  and  the  process  and  product 
approach  results. 

•  The  preaward  survey  is  adequate  for  its  intended  use.  DoD  agencies  must 
cooperate  better  to  make  the  system  work  more  effectively. 

14.  Question  14.  (DoD  and  Contractor  Quality  Assurance  Relationship) 

In  your  view  has  the  DoD  culture  and  approach  changed  in  quality  assurance 
audit  procedures  under  accepting  ‘most  effective  quality  system’  instead  of  MIL-Q-9858? 

a.  Discussion 

The  culture  of  DCMA  from  an  industry  viewpoint  is  addressed  in  answering  this 
question.  The  question  was  to  gain  perspective  on  outsiders’  view  of  DCMA  culture  and 
attitudes  effecting  MIL-Q  to  ISO  change. 

b.  Reponses 

•  The  QARs  appear  to  have  not  changed  at  all. 

•  Yes,  overall  the  transition  by  DOD  to  the  “industry  standard”  philosophy 

is  in  place. 

•  Our  relationship  with  DCMA  is  better  now  than  it  has  been  in  the  past. 
The  process  improvements  have  been  good.  But  as  DCMA  reduced  their 
personnel  by  two-thirds  over  the  past  decade  all  you  had  left  was  older, 
more  senior  employees.  These  employees  come  from  a  past  culture  of 
inspection  and  monitoring  that  we  (DCMA)  cannot  trust  the  contractor. 
You  reach  a  point  where  it  is  very  difficult  for  DCMA  to  change  the 
culture.  It  is  a  generational  issue.  They  (DCMA)  are  being  taught,  trained 
and  told  to  do  it  differently  but  can’t  or  won’t  change  due  to  the  past 
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cultural  influence.  The  environment  has  become  increasingly  more 
technical  with  change  occurring  more  rapidly  than  in  the  past.  With  the 
high  cut-backs  in  personnel  and  few  new  people  it  is  difficult  for  DCMA 
to  attract  new  people  with  the  qualifications  and  skills  required  to 
adequately  evaluate  an  organization  from  a  systems  and  technical 
approach. 

•  The  change  is  for  the  better.  The  culture  is  focusing  more  to  the  processes 
associated  with  quality  than  the  end  result  inspections  of  the  past. 

15.  Question  15.  (DoD  and  Contractor  Quality  Assurance  Relationship) 

How  effective  (strength  and  weaknesses)  is  the  quality  assurance/ management 
audit  and  later  surveillance  performed  by  ISO  Registrars  and  DCMA  or  DCAA  Quality 
Assurance  Representatives? 

a.  Discussion 

This  question  focused  on  the  audit  procedures,  rather  than  the  auditors,  and  the 
audits  strengths  and  weaknesses. 

b.  Responses 

•  The  ISO  registrars  are  reviewing  the  system  every  six  months;  this 
frequency  is  greater  than  the  QARs  review.  My  opinion  is  the  QARs 
would  wait  for  a  nonconformance  to  occur  and  then  review  a  process, 
which  is  to  late.  The  ISO  registrars  are  more  proactive. 

•  Not  nearly  as  effective  as  our  own  internal  reviews.  They  lack  the 
understanding  of  the  processes  and  cannot  therefore  adequately  judge 
them.  They  are  not  intended  to  identify  all  issues.  (DCMA)  audit  can  and 
should  only  be  cursory  in  nature.  The  supplier  must  earn  trust.  (Supplier 
trust)...  provides  the  confidence  necessary  to  the  customer.  It  is  the 
supplier’s  responsibility  to  prove  that  risk  is  minimal  and  good  value  is 
being  delivered  in  each  case.  The  systems,  which  are  implemented,  to 
comply  with  ISO  are  but  one  step  in  creating  that  assurance. 

•  Our  DCMA  auditors  perform  joint  audits  with  us,  as  a  single  team.  I 
believe  that  the  DCMA  participation  in  this  effort  is  strong. 

•  The  registrars  are  head- and- shoulder  above  DCMA  on  their  ability  to 
evaluate  compliance  to  the  ISO  Standard.  They  (ISO)  are  much  more 
capable  to  understand  the  systems  evaluation  of  an  organization  and  the 
organizations’  written  procedures  to  match  the  system.  Registrars  look  at 
the  entire  business  system.  Registrars  are  not  very  good  at  looking  at  a 
product  and  measuring  its  conformity  and  compliance  to  the  procedures. 
DCMA  is  hardware  oriented  and  much  more  familiar  with  the  hard  ware 
requirements. 
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•  The  biggest  disconnect  is  between  the  DCMA  and  the  system  user.  More 

needs  to  be  done  by  DCMA  to  better  understand  and  determine  what  the 
user  is  looking  for.  This  places  the  contractor  as  the3d  party  police  officer 
between  the  DCMA  and  the  user.  The  DCMA  lacks  insight  from  the  user 
and  the  user’s  requirements  and  how  to  meet  the  user’s  expectations. 

16.  Question  16.  (DoD  and  Contractor  Quality  Assurance  Relationship) 

What  changes  in  training  should  DoD  make  to  better  address  ISO  9000:2000  and 
better  address  the  quality  assurance  needs  of  industry  and  DoD  customer  demands? 

a.  Discussion 

This  question  was  meant  to  solicit  input  from  industry  on  how  best  to  train 
DCMA  representatives  conducting  quality  assurance  auditing  and  surveillance. 

b.  Responses 

•  Provide  training  to  the  QARs  for  the  new  standard 

•  I  think  the  DoD  made  the  correct  decision  when  they  decided  to  get  out  of 
the  standards  business  and  allow  industry  to  run  and  maintain  the  system. 

•  Perhaps  sending  QARs  to  the  same  training  used  in  industry  is  a  good 
suggestion?  The  American  Society  for  Quality  has  excellent  training 
programs  for  auditors  and  technicians  that  would  benefit  the  QARs. 

•  I  think  that  DoD  should  have  their  auditors  trained  by  RAB  certified 
trainers  (just  like  most  of  the  industry  does),  and  not  receive  their  training 
from  Government  sources. 

•  It  seems  to  me  DCMA  is  pushing  away  from  the  hardware  and  product 
audit  and  focusing  more  on  the  systems  approach.  DCMA  should  stay 
away  from  this  when  ISO  has  registered  an  organization  since  ISO  looks 
at  the  quality  or  management  system  overall.  DCMA  should  remain 
focused  on  the  hardware  and  products.  DCMA  is  the  representative  of  the 
military  customer  and  user  and  should  focus  on  the  product. 

•  DCMA  focus  should  be  the  procurement  and  risk  management  of  the 
contract.  DCMA  is  a  tool  for  the  Government  to  use  to  control  the  risk 
went  the  Government  buys  something.  The  question  is  how  does  the 
Government  want  to  use  DCMA.  ISO  as  a  tool  determines  if  an 
organization  has  the  capability  from  a  systems  point  of  view.  DCMA 
primarily  inspects  hardware  once  the  order  is  in  and  not  a  very  good  tool 
to  determine  the  future  risk  of  the  supplier.  DCMA  seems  more  effective 
after  the  market  research  is  conducted  and  a  supplier  is  selected.  DCMA 
uses  in  process  and  final  inspection  to  compare  the  product  against 
specifications.  This  type  of  inspection  is  better  on  tank  tread  than  on  a 
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missile  guidance  system.  How  to  use  DCMA  depends  on  the  product  and 
how  best  to  mitigate  risk. 

•  Continue  ISO  and  current  industry  quality  assurance  training.  I  see 
DCMA  weakness  is  their  inconsistent  approach  between  the  product 
quality  and  the  quality  of  the  entire  organizations  systems  and  processes. 
ISO  looks  at  a  whole  product  line.  DCMA  must  take  a  more  process 
oriented  view  because  all  our  process  are  finally  linked  to  the  product. 
DoD  organizations  could  become  Third-party  ISO  registered 

17.  Question  17.  (DoD  and  Contractor  Quality  Assurance  Relationship) 

How  is  the  cost  associated  with  becoming  ISO  9000  certified  applied  to  industry 
cost  of  doing  business  (CAS  and  GAAP)? 

a.  Discussion 

Current  CAS  does  not  allow  industry  to  directly  charge  ISO  QMS  application  to 
its  contracts.  The  question  was  asked  to  gamer  responses  of  how  does  industry  apply  the 
ISO  Registration  and  QMS  to  that  of  doing  business. 

b.  Responses 

•  From  a  business  perspective  one  would  be  hard  pressed  to  associate  the 
initial  outlays  to  the  benefits  over  time,  although  estimates  could  be  put 
together  for  ROI.  Most  businesses  considering  ISO  certification  are  doing 
so  because  of  a  need  expressed  by  their  customers  and  certification  is 
pursued  to  ensure  that  a  competitive  edge  is  maintained,  or  simply  to 
maintain  their  current  business.  In  many  cases  a  business  can  find  itself 
non-competitive  in  international  or  certain  domestic  markets  if  it  does  not 
have  and  maintain  a  certified  quality  system. 

•  The  cost  of  registration  is  an  overhead  expense  and  is  allocated  across  the 
entire  business.  (I  am  not  familiar  with  CAS/GAAP) 

•  The  fee  for  registration  is  a  significant  amount  of  money  to  give  to  the 
registrar.  ISO  cost  is  expensive  and  that  expense  is  passed  down,  in  one 
form  or  another,  in  overhead  or  what  the  customer  pays.  It  goes  to  profit, 
an  organization  must  cover  their  cost  and  it  will  end  up  somewhere  in  the 
profit  line  whether  it  is  covered  as  overhead  or  as  another  cost. 
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c. 


DCMA  INTERVIEW  RESPONSES 


1.  Question  1  (ISO  9000:2000) 

How  is  DCMA  changing  their  approach  to  quality  assurance  and  management 
certification  and  monitoring  due  to  the  changes  in  ISO  9000  1994  to  2000,  as  it  applies  to 
higher- level  quality  standards  and  risk  management  associated  in  this  area? 

a.  Discussion 

This  question  was  designed  for  the  respondents  to  provide  insight  to  the  reader  on 
the  changes  DCMA  is  making  in  addressing  ISO  changes. 

b.  Responses 

•  The  DCMA  approach  has  not  significantly  changed.  The  2000  ISO 

versions  are  seen  and  addressed  as  any  other  higher- level  quality 
requirement.  DCMA  is  currently  reviewing  its  practice  of  issuing 
statements  of  qualification  regarding  supplier  higher- level  quality 
compliance  with  the  intent  of  deleting  the  practice,  however  the 
determination  is  not  based  on  the  ISO  change,  but  rather  on  the  DOD 
practice  of  not  requiring  certified  systems. 

•  DCMA  Headquarters  view  is  (ISO  9000)  2000  is  not  significantly 

different  from  (ISO  9000)  1994.  The  civilian  sectors  see  this  differently 
and  are  committing  resources  to  make  the  change.  DCMA  should  follow 
suite,  at  least  partially,  to  assure  our  auditors  are  able  to  perform  an  audit 
at  an  appropriate  level  of  knowledge  and  expertise. 

2.  Question  2  (ISO  9000 :2000) 

Have  the  problems,  shortcomings,  and  lessons  learned  with  the  previous  ISO 
9000  standards  and  registration  been  addressed  under  ISO  9000:2000? 

a.  Discussion 

This  question  was  used  to  allow  organizations  to  provide  the  researcher  with 
insight  into  what  DCMA  quality  assurance  practitioners  saw  as  the  strengths, 
weaknesses,  and  opportunities  of  both  ISO  9000:1994  and  2000  provide. 

b.  Responses 

•  Both  respondents  believed  the  new  version  addresses  shortcomings  with 

customer  satisfaction  and  continual  improvement  but  introduces  new 

problems  and  concerns  due  to  the  cancellation  of  ISO  9002  and  9003  for 
industry,  registers  and  compliance  activities.  DCMA  plays  a  unique  role 
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when  addressing  the  shortcomings  of  any  QMS.  As  the  frontline  worker 
they  are  enabled  to  address  issues  of  this  type.  DCMA,  as  an  in-plant 
representative,  is  the  ombudsman  for  the  Governments  quality  assurance 
and  management  program  applied  to  procurement.  As  the  frontline 
ombudsmen  the  DCMA  representative  protects  Government  interests  in 
product  assurance  matching  customer  requirements  with  production 
challenges  face  by  industry. 

3.  Question  3  (ISO  9000:2000) 

Will  their  be  any  changes  to  the  FAR,  DoDD  5000,  and  to  DCMA  preaward 
surveys  and  surveillance?  If  so,  can  you  provide  an  overview  or  any  documents  showing 
what  these  changes  might  be? 

a.  Discussion 

The  question  was  asked  to  update  any  recent  changes  to  DoD  guidance  not 
covered  under  Chapter  III. 

b.  Responses 

•  Unaware  of  any  changes  proposed  solely  due  to  the  transition.  A  FAR 

change  was  contemplated  to  introduce  ISO  9001-2000,  however  the 
current  version  of  the  FAR  does  not  date  versions  so  the  2000  version  is 
already  contained.  A  FAR  change  has  been  drafted  that  will  include  the 
designation  of  ANSI/ISO/ASQ  Q9001  as  a  higher-level  quality 
requirement.  This  change  intended  to  assure  that  the  accepted  American 
version  of  ISO  9001  is  included. 

•  We  are  currently  working  on  an  ISO  9000-2000  checklist  in  DCMA.  Our 
checklist  is  nearly  identical  to  most  registrars  except  for  about  10%  to 
20%  of  the  document.  In  reviewing  many  registrars  checklist  we  say  that 
the  majority  of  the  documents  was  the  same.  The  similarities  are  in  the 
“shall”  and  “will”  clauses  of  the  registrars’  checklists.  The  remainder  of 
the  registrars’  checklist  focused  on  their  interpretation  of  the  “may”  and 
“should”  clauses  of  ISO.  The  DCMA  ISO  9000-2000  checklist  is  based 
solely  on  the  “shall”  and  “will”  clauses.  DoD  cannot  contractually  hold  an 
organization  to  the  “should”  clauses  unless  they  are  specified  outside  the 
of  the  ISO  9000:2000  requirement. 

4.  Question  4  (ISO  9000:2000) 

What  planning,  guidance,  and  training  is  DCMA  conducting  to  address  the  ISO 
9000:2000  changes? 
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a.  Discussion 

This  question  was  used  to  ascertain  the  particular  local  and  agency  wide  framing 
on  ISO  9000:2000. 

b.  Responses 

•  The  change  has  been  introduced  to  the  workforce  and  the  deltas  between 

the  1994  and  2000  versions  provided.  Under  a  decentralized  training 
environment,  introductory  training  has  been  provided  to  a  number  of 
assigned  personnel.  Additionally,  local  organizations  are  encouraged  to 
obtain  more  intensive  training  as  required. 

•  Training  is  limited.  Funding  is  limited  to  non-existent  depending  on  the 

command.  CMO’s  are  told  to  fund  training  for  ISO  from  their 
discretionary  budget.  The  focus  is  on  delta  training  to  get  the  most  effect 
for  the  limited  resources  for  training.  The  training  philosophy  across 
DCMA  organization  to  get  our  workforce  knowledgeable  on  the  2000 
version  is  “train  the  trainer.”  Many  commands  are  sending  one  or  two 
people  to  training  and  in-turn  they  are  responsible  for  training  the  people 
in  their  local  organization.  There  are  some  weaknesses  in  this  approach 
but  given  the  current  resource  constraint  it  is  the  current  course  of  action. 

•  Under  the  MI-Q  9858  and  45208,  each  came  with  a  handbook,  H50  and 
H51  respectively,  on  how  to  properly  use  the  standard.  DCMA  publishes 
no  such  handbook  for  ISO,  nor  does  the  IOS.  IOS  mandates  what  the 
standard  is,  not  how  to  implement  or  evaluate. 

5.  Question  5  (ISO  9000:2000) 

From  your  perspective  what  changes  in  ISO  9000  provide  the  greatest  advantage 
and/or  disadvantages  to  industry  and  the  DoD  procurement? 

a.  Discussion 

This  question  was  asked  to  establish  DCMA  respondents  opinion  on  the 
advantages/disadvantages  of  ISO  9000:2000 

b.  Responses 

Overall  opinion  was  the  introduction  of  exclusions  to  the  requirements  contained 
in  clause  7  provides  both  the  greatest  advantages  and  disadvantages  to  both  industry  and 
the  Government.  The  concept  allows  for  the  true  commercialization  of  the  QMS; 
however  industry  is  responsible  for  truly  understanding  their  markets  and  customers  and 
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the  DoD  is  now  responsible  for  understanding  the  exclusions  principles  and  determining 
if  exclusions  are  appropriate,  both  in  the  pre  and  post  contract  award  phases. 

6.  Question  6  (ISO  9000:2000) 

How  does  or  what  role  does  customer  satisfaction  and  contractor  past 
performance  play  in  DCMA  quality  assurance  and  management?  What  are  its  problems 
and  how  could  it  be  made  better? 

a.  Discussion 

DCMA  has  the  unique  role  of  providing  the  most  interfaces  with  industry  as  the 
DoD  representative;  this  question  was  used  to  gain  DCMAs  perspective  on  that  role  as  it 
pertains  to  customer  satisfaction. 

b.  Responses 

Customer  satisfaction  and  supplier  past  performance  serves  a  great  role  in  DCMA 
QA  activities.  Risk  management  activities  are  extremely  reliant  on  these  factors. 

Problems  revolve  around  communications  and  Government  reluctance  to  export  past 
performance  information,  at;  I  believe  the  insistence  of  industry. 

7.  Question  7  (ISO  9000:2000) 

Should  the  redundancies  between  ISO  9000  registration  and  periodic  review 
compared  to  DCMA  preaward  survey  and  surveillance  be  eliminated?  Why  or  why  not 
should  Government  evaluate  the  redundant  areas? 

a.  Discussion 

This  question  was  posed  to  address  one  of  the  major  problems  associated  between 
the  DCMA,  industry,  and  registrars  of  ISO.  The  questions  goal  was  to  bring  into  context 
the  Third-party  registrar  and  DCMA  role  in  quality  assurance  audit  and  surveillance  and 
provide  DCMA  responses  to  Industry  Question  Number  11. 

b.  Responses 

Redundancies  should  be  reduced;  DCMA  policy  requires  that  first,  second  and/or 
third  party  audit  results  be  assessed  and  used  prior  to  performance  of  redundant  activities. 
I  do  not  believe  that  DCMA  activities  relative  to  suppliers  QMS  are  redundant.  A  case 
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possibly  can  be  made  that  preaward  surveys  regarding  QA  capabilities  are  not  necessary 
for  registered  suppliers,  but  that  registration  should  be  reviewed  in  lieu. 

8.  Question  8  (ISO  9000:2000) 

Has  the  SPI  and  the  move  to  best  commercial  standards,  like  ISO,  provided  the 
DoD  a  better  product  and  made  the  acquisition  process  better  for  customer,  DoD 
manager,  and  industry? 

a.  Discussion 

This  question  was  used  to  gain  the  past  experience  of  the  DCMA  interviewees 
and  draw  some  conclusions  about  effectiveness  of  SPI. 

b.  Responses 

•  The  SPI  of  only  one  standard  to  industry  is  deceiving.  MIL-Q  and  MIL- 1 

were  not  canceled.  They  are  not  the  preferred  method  according  to  DoD 
policy.  So  when  we  (DoD)  stopped  dictating  what  quality  system  to  use  to 
allow  the  “most  effective  quality  system”  we  opened  the  door  for  industry 
to  choose  the  system.  Many  organizations  decided  to  remain  with  the 
MIL-Q  and  MIL-I  standards.  DCMA  now  must  not  only  understand  ISO 
9000:2000  but  all  of  the  previous  Military  standards  still  in  use,  many 
industry  accepted  quality  standards  like  QS  9000  the  automobile  industry 
uses,  as  well  as,  ISO  9000:1994.  The  way  the  FAR  is  written  we  are 
discouraged  to  dictate  to  an  organization  any  of  these  systems  as  long  as 
they  meet  the  “most  effective”  criteria.  The  challenge  of  understanding  all 
of  these  systems,  although  there  are  many  similarities  between  them,  is 
difficult  for  DCMA  to  consistently  and  appropriately  manage  across  the 
numerous  sectors  of  business  DoD  is  contracted  with. 

•  Theoretically  it  makes  the  acquisition  process  better. 

•  DoD  suppliers’  complaint  about  the  cost  of  having  two  or  more  systems 
prior  to  SPI  is  only  a  half-truth.  Most  organizations  only  used  one  system, 
MIL-Q,  the  cost  of  quality  is  something  an  organization  will  commit  to 
even  if  it  is  not  contractually  outlined.  The  cost  of  implementing  ISO,  if 
they  chose  to  leave  MIL-Q,  is  only  a  5%  to  10%  increase.  The  similarity 
between  ISO  9000-1994  and  MIL-Q  9858  were  not  significant  enough,  in 
my  opinion,  to  warrant  significantly  more  cost.  Suppliers  wanted  DoD 
inspectors  out  of  their  factories  and  in  SPI  industry  saw  a  way  to 
accomplish  the  elimination  of  in-house  DCMA  representatives.  Industry 
and  NATOs  push  0  use  ISO  instead  of  the  Allied  Quality  Assurance 
Procedures  (AQAPs)  pushed  DoD  to  make  the  change. 
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9.  Question  9  (Quality  System  Versus  Quality  Product) 

One  criticism  of  ISO  9000  registration  is  it  provides  no  guarantee  of  product 
quality  or  conformity,  only  that  a  quality  process  is  in  place.  Do  you  agree,  disagree  with 
this  statement? 


a.  Discussion 

This  question  is  based  on  the  recent  incidents  of  ISO  9000  registered 
organizations  producing  non- conforming  and  defective  material.  Its  purpose  was  to  elicit 
comments  on  some  of  the  problems  or  misconceptions  about  ISO  registration  and  what 
measures  can  be  taken  to  increase  quality  control  and  product  conformance. 

b.  Responses 

•  There  are  good  registrars  and  poor  ones.  Some  will  register  any 
organization  given  the  right  price.  Most  do  not  and  do  conduct  a  good 
quality  system  Third  Party  registration  against  ISO.  What  Third  Party  ISO 
registration  does  for  us  (DCMA)  is  to  provide  a  certain  level  of  confidence 
in  the  suppliers  system  we  can  use  to  assess  the  risk  of  the  potential 
contract  and  make  an  evaluation  on  how  to  precede  with  the  DCMA 
preaward  survey.  Registrars  look  at  the  quality  system  as  a  whole  DoD 
should  not  only  look  at  the  total  system  but  below  that  level  as  well. 
DCMA  must  inspect  or  provide  insight  into  the  product  process  quality. 
We  track  forward  and  back  on  product  development. 

•  There  are  no  absolute  guarantees  that  either  will  result  in  a  quality 
product.  They  should  assist  in  identifying  any  system  discrepancies  that 
may  lead  to  sub- standard  product  quality. 

10.  Question  10  (Quality  System  Versus  Quality  Product) 

What  quality  assurance/management  measures  does  DCMA  take  to  insure  a 
supplier  demonstrates  product  conformity  and/or  critical  characteristics,  in  terms  of 
defect  levels,  are  acceptable;  and  that  improvements  are  planned  and  later  demonstrated? 

a.  Discussion 

The  same  question  type  was  asked  of  industry  to  see  what  additional  measures  are 
used  to  provide  product  quality  assurance. 
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b.  Responses 

DCMA  employs  a  risk  management  philosophy  of  identifying  key  processes  and 
their  potential  risks  to  performance,  cost  and  schedule.  Surveillance  techniques, 
particularly  regarding  performance  risk,  include  assessments  of  QMS,  analysis  of 
processes,  audits  of  products  and  ongoing  data  analysis. 

11.  Question  11  (DoD  and  Contractor  Quality  Assurance  Relationship) 

Has  ISO  9000  become  the  quality  management  standard  for  DoD  contractors,  i.e., 
must  a  contractor  become  ISO  9000  certified  to  conduct  business  with  the  DoD 
especially  in  contracts  requiring  a  higher-  level  quality  standard? 

a.  Discussion 

The  issue  the  researcher  wanted  to  raise  in  this  question  is,  although  ISO  is  not 
mandated  by  any  Federal  procurement  regulation  and  guidance,  has  ISO  influence 
created  an  environment  in  which  industry  must  be  compliant  with  ISO  to  conduct 
business. 

b.  Responses 

•  Under  A76  the  services  have,  in-essence,  become  responsible  again  for 

contract  administration.  Under  an  A-76  program  the  local  procuring  office 

administers  the  contract.  Local  procuring  agencies  use  local  policy.  The 
policy  is  usually  based  on  Army,  Navy,  or  Air  Force  Regulation  and 
Guidance,  not  ISO  standards.  An  organization  like  DynaCorps  Inc.  who  is 
ISO  Third  Party  Registered  and  accepts  an  A 76  still  has  to  be  judged 
against  the  local  policy  for  award  and  later  for  surveillance.  Some  local 
policy  has  adapted  ISO.  Service  members  and  local  contract  officers  have 
trained  to  conduct  ISO  Third-Party  Registration.  One  example  is  DSC- 
Columbus.  A76  is  another  product  of  inconsistency  of  quality  assurance 
and  management  across  DoD  faced  by  industry. 

12.  Question  12  (DoD  and  Contractor  Quality  Assurance  Relationship) 

Should  DoD  impose  ISO  9000  standards  and  registration  on  contractors  and  sub¬ 
contractors  as  an  entrance  to  conducting  business  with  the  DoD? 

a.  Discussion 

This  question  was  meant  to  elicit  general  comments  on  the  benefits  and  problems 
of  a  DoD  mandate  of  ISO  9000. 
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b.  Responses 

It  was  the  opinion  of  the  DCMA  respondents  that  ISO  should  not  be  imposed. 
DCMA  as  well  as  industry  believed  that  it  will  limit  competition  and  may  force  out 
smaller  entities.  Further  analysis  is  providing  under  Industry  Question  Number  10. 

13.  Question  13  (DoD  and  Contractor  Quality  Assurance  Relationship) 

Some  DoD  Quality  Assurance  Representatives  (QARs)  complain  some  ISO 
registered  companies  believe  that  if  they  are  ISO  registered  that  only  limited  DoD 
inspection  is  required.  Should  ISO  registration  be  accepted  by  DoD  QARs  as  certification 
for  the  Preaward  Quality  Assurance  Survey  and  has  registration  reduced  and/or 
eliminated  DoD  on-site  quality  inspections  or  audits  during  the  surveillance  phase,  should 
it? 


a.  Discussion 

This  question  was  posed  to  address  one  of  the  major  problems  associated  between 
the  DCMA,  industry,  and  registrars  of  ISO.  The  questions  goal  was  to  bring  into  context 
the  Third-party  registrar  and  DCMA  role  in  quality  assurance  audit  and  surveillance. 

b.  Responses 

•  A  case  may  be  made  to  limit  preaward  surveys  of  companies  that  have 
certification;  I  believe  more  study  is  needed.  I  don’t  believe  that  ISO 
registration  has  been  responsible  for  reduced  QA  activities.  It’s  my 
opinion  that  QA  activities  on-site  have  been  reduced  due  to  personnel  and 
money  constraints  forcing  “smarter”  decisions  at  both  the  pre  and  post 
award  phases  to  cone  entrate  activities  based  on  risk. 

•  Earlier  each  service  had  their  respective  folks  doing  contract 

administration,  i.e.,  AF  PROS,  and  NAV  PROS.  We  consolidated 
resources  and  became  DCMC  under  DLA.  We  recently  became  our  own 
agency  and  no  longer  fall  under  DLA.  Our  early  auditors  used  the  8200.1 
“Black  Book.”  The  “Black  Book”  spelled  out  how  to  conduct  quality 
audits.  The  book  used  defined  and  express  verbiage  for  many  contracting 
scenarios  and  issues.  We  use  product  evaluations  (PEs)  and  Product 
Verification  Inspections  (PVIs)  in  the  conduct  of  contract  administration. 
The  1990’s  saw  the  end  of  this  type  of  inspection  process  and  ushered  in 
insight  instead  of  inspection.  The  “One  Book”  replaced  the  “Black  Book” 
and  with  it  went  specifics  to  generalists.  DCMA  went  from  a  stovepipe 
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organization  of  the  Quality  Directorate,  the  Engineering  Directorate,  and 
the  Contracting  Directorate  to  a  teaming  approach.  The  approach  does  not 
work  well  and  comes  with  its  own  set  of  problems.  Under  the  stovepipe 
approach  DCMA  was  able  to  mandate  and  control  policy  under  each  of  the 
directorates.  The  team  leader  now  controls  the  agenda  of  the  team.  The 
team  leaders  are  not  usually  trained  in  all  the  functions  so  what  directorate 
the  team  leader  came  from  is  the  main  agenda  of  the  team  with  the  other 
areas  usually  suffering  due  to  the  teaming  approach.  Under  the  new 
teaming  system  a  great  deal  of  latitude  is  given  to  each  DCMA  command 
and  even  office  to  define  quality  management  and  assurance  according  to 
current  contract  preaward  surveys.  The  approach  causes  great 
inconsistency  between  how  DCMA  conducts  audits  and  inspection,  not 
only  regionally  but  also  with- in  the  same  office. 

14.  Question  14  (DoD  and  Contractor  Quality  Assurance  Relationship) 

In  your  view  has  the  DoD  culture  and  approach  changed  in  quality  assurance 
audit  procedures  under  accepting  ‘most  effective  quality  system’  instead  of  MIL- Q- STD 
9858? 


a.  Discussion 

The  culture  of  DCMA  from  an  employee’s  viewpoint  is  addressed  in  answering 
this  question.  The  question  was  to  gain  perspective  on  an  insiders’  view  of  DCMA 
culture  and  attitudes  effecting  MIL-Q  to  ISO  change. 

b.  Responses 

•  No,  not  particularly,  due  to  the  “most  effective  quality  system”  concept. 

•  In  1988  DoD  wanted  nothing  to  do  with  ISO.  The  culture,  while  slowly 

changing,  still  has  yet  to  wholly  embrace  the  SPI  ideals  around  ISO. 
Culture  is  difficult  to  change.  The  aging  of  our  workforce,  the  decline  in 
our  quality  assurance  workforce  numbers  from  24,000  in  1995  to  12,000 
currently  (numbers  are  approximate)  and  our  inability  to  attract  some  of 
the  best  talent  in  the  field  of  quality  assurance  and  management  are  factors 
affecting  the  DCMA  culture. 
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